-- Logs begin at Mon 2022-07-18 13:27:27 UTC, end at Mon 2022-07-18 14:36:02 UTC. -- Jul 18 13:54:14.364726 dv-cinder-849846-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 18 13:54:14.376536 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 18 13:54:14.378496 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 18 13:54:14 2022] *** Jul 18 13:54:14.378496 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 18 13:54:14.378496 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 18 13:54:14.378496 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: nodename: dv-cinder-849846-1-iscsi Jul 18 13:54:14.378496 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: machine: x86_64 Jul 18 13:54:14.378496 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: clock source: unix Jul 18 13:54:14.378496 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: pcre jit disabled Jul 18 13:54:14.378496 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: detected number of CPU cores: 4 Jul 18 13:54:14.378496 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: current working directory: / Jul 18 13:54:14.378496 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: detected binary path: /usr/bin/uwsgi-core Jul 18 13:54:14.379129 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: your processes number limit is 47680 Jul 18 13:54:14.379129 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: your memory page size is 4096 bytes Jul 18 13:54:14.379129 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: detected max file descriptor number: 2048 Jul 18 13:54:14.379129 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: lock engine: pthread robust mutexes Jul 18 13:54:14.379129 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: thunder lock: enabled Jul 18 13:54:14.379129 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 18 13:54:14.379129 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 18 13:54:14.418955 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: Python main interpreter initialized at 0x55bbf06139c0 Jul 18 13:54:14.418955 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: python threads support enabled Jul 18 13:54:14.418955 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: your server socket listen backlog is limited to 100 connections Jul 18 13:54:14.418955 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: your mercy for graceful operations on workers is 90 seconds Jul 18 13:54:14.419323 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: mapped 403077 bytes (393 KB) for 2 cores Jul 18 13:54:14.419397 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: *** Operational MODE: preforking *** Jul 18 13:54:14.419659 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: *** uWSGI is running in multiple interpreter mode *** Jul 18 13:54:14.419659 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: spawned uWSGI master process (pid: 80147) Jul 18 13:54:14.419868 dv-cinder-849846-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 18 13:54:14.422417 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: spawned uWSGI worker 1 (pid: 80158, cores: 1) Jul 18 13:54:14.425668 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: spawned uWSGI worker 2 (pid: 80160, cores: 1) Jul 18 13:54:14.425668 dv-cinder-849846-1-iscsi devstack@keystone.service[80147]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 18 13:54:19.848788 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 13:54:19.848788 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: from cryptography.utils import int_from_bytes Jul 18 13:54:19.849570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 13:54:19.849570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: from cryptography.utils import int_from_bytes Jul 18 13:54:19.850838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 13:54:19.850838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: from cryptography.utils import int_from_bytes Jul 18 13:54:19.851291 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 13:54:19.851291 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: from cryptography.utils import int_from_bytes Jul 18 13:54:19.897797 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 13:54:19.897984 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 13:54:19.898175 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] command line args: [] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 13:54:19.898518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 13:54:19.898695 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ================================================================================ {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 13:54:19.898809 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 13:54:19.898945 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] admin_token = **** {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.899026 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 13:54:19.899185 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] config_dir = [] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.899265 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] command line args: [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 13:54:19.899373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.899459 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 13:54:19.899543 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] config_source = [] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.899621 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ================================================================================ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 13:54:19.899715 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] debug = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.899882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] admin_token = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.899981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.900229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] config_dir = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.900340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.900438 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.900541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.900638 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] config_source = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.900742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] debug = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.900839 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.900941 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.901066 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.901164 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.901280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] list_limit = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.901378 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.901479 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] log_config_append = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.901479 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.901665 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.901761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] log_dir = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.901866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.901963 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] log_file = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.902064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.902180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.902259 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_config_append = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.902376 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.902460 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.902460 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_dir = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.902598 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.902717 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_file = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.902798 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.902874 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.903002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.903002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.903187 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.903262 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.903378 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.903484 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.903574 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.903716 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.903796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.903913 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.904311 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.904311 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.904547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.904662 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.904775 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.904892 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.905011 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.905138 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.905254 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.905254 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.905491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.905615 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.905615 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.905845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.94/identity {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.905845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] publish_errors = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.906064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.906177 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.906300 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.906424 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.906424 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.906621 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.94/identity {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.906736 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.906861 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] publish_errors = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.906984 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.907109 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.907234 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] standard_threads = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.907368 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.907496 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.907612 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.907727 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.907841 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.907977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.908151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.908276 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] use_journal = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.908276 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] use_json = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.908506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] standard_threads = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.908628 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] use_stderr = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.908750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.908877 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] use_syslog = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.908994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.909110 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.909224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.909339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.909465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_journal = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.909582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.909702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_json = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.909702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_stderr = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.909923 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.910044 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_syslog = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.910169 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.910286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 13:54:19.910407 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.910522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.910643 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.910764 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.910888 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.911004 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.911120 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.911240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.911361 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.911477 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.911603 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.911719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.911834 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.911972 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.912091 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.912217 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.912342 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.912459 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.912574 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.912688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.912809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.912925 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.913055 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.913176 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.913293 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.913407 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.913521 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.913636 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.914349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.914486 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.914604 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.914719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.914842 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.914964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.915089 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.915207 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.915335 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.915451 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.915566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.915687 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.915811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.915928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.916072 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.916072 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.916288 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.916403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.916533 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.916656 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.916772 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.916887 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.917002 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.917117 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.917245 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.917368 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.917485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.917599 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.917714 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.917837 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.917953 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.918068 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.918182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.918303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.918424 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.918553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.918670 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.918784 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.918899 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.919019 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.919141 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.919268 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.919390 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.919512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.919629 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.919744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.919859 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.920071 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.920191 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.920313 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.920429 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.920544 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.920659 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.920782 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.920904 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.921019 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.921134 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.921248 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.921363 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.921489 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.921606 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.921720 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.921835 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.921955 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.922070 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.922193 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.922314 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.922436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.922534 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.922635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.922731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.922834 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.922931 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.923033 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.923143 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.923245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.923245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.923473 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.923591 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.923696 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.923810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.923926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.924093 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.924222 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.924341 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.924456 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.924567 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.924694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.924694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.924910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.925036 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.925152 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.925276 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.925393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.925497 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.925611 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.925735 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.925855 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.925970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.926095 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.926212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.926328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.926438 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.926562 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.926679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.926795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.926917 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.927023 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.927138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.927266 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.927266 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.927486 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.927607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.927729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.927854 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.927990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.928108 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.928223 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.928349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.928467 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.928580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.928703 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.928825 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.928942 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.929058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.929173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.929286 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.929408 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.929525 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.930202 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.930323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.930440 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.930565 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.930684 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.930801 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.930917 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.931033 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.931149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.931149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.931378 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.931495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.931611 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.931733 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.931850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.932063 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.932194 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.932312 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.932429 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.932545 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.932661 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.932661 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.932899 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.933028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.933147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.933253 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.933368 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.933484 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.933484 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.933710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.933828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.933943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.934060 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.934182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.934308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.934434 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.934551 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.934673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.934790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.934905 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.935028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] auth.external = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.935153 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.935272 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.935388 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.935504 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.935621 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.935745 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.935745 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] auth.password = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.936492 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.936609 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] auth.token = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.936708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.936904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.937046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.937156 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.937263 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.937391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.937391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.937587 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.external = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.937685 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.937792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.937901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.938009 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.938110 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.938207 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.938305 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] credential.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.938305 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.938501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.password = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.938501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.token = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.938692 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.938789 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.938886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.938886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.939082 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.939181 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.939278 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.939382 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.939480 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.939577 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.939683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.939788 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.939886 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.940014 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.940121 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.940121 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.940323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.940423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.940521 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.940619 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.940716 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.940865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.940978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.941100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.941217 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.941334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.941451 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.941569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.941697 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] federation.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.941697 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.941920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.942038 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.942156 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.942281 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.942407 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.942526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.942645 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.942761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.942879 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.942996 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.943121 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.943241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.943348 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.943453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.943553 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.943650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.943760 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.943860 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.943989 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.944247 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.944348 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.944446 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.944557 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.944659 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.944757 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.944854 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.944952 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.945056 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.945452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.945567 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.945667 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.945789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.945907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.946025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.946226 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.946347 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.946464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.946581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.946698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.946826 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.946946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.947063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.947180 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.947298 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.947298 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.947528 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.947651 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.947770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.947889 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.948028 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.948148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.948279 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.948279 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.949077 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.949201 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.949319 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.949447 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.949568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.949686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.949804 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.949922 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.950040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.950166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.950286 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.950404 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.950530 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.950649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.950767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.950767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.951056 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.951056 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.951540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.951657 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.951774 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.951899 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.952044 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.952164 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.952282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.952398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.952515 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.952645 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.952766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.952885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.953002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.953119 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.953236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.953362 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.953481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.953598 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.953716 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.953834 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.953958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.954084 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.954237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.954357 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.954474 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.954474 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.954692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.954817 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.954943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.955061 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.955061 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.955282 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.955399 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.955527 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.955646 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.955768 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.955886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.956103 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.956223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.956354 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.956473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.956591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.956707 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.956825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.956942 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.957067 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.957187 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.957306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.957423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.957543 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.957668 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.957795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.957914 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.958031 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.958149 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.958266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.958385 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.958510 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.958662 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.958784 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.958902 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.959018 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.959155 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.959285 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.959406 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.959542 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.959662 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.959780 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.959905 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.960058 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.960178 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.960294 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.960423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.960542 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.960668 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.960811 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.960929 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.961046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.961163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.961280 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.961406 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.961527 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.961644 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.961760 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.961760 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.961981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.961981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.962217 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.962217 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.962437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.962555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.962673 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.962798 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.962918 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.963035 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.963152 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.963268 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.963385 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.963511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.963631 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.963749 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.963866 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.963996 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.964115 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.964245 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.964365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.964365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.964586 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.964703 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.964819 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.964945 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.965063 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.965181 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.965298 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.965415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.965532 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.965659 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.965778 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.965895 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.966012 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.966129 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.966245 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.966370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.966489 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.966606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] resource.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.966724 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.966841 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.966958 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.967083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.967202 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.967320 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.967437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.967558 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.967676 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.967803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.967922 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.968057 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.968175 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.968292 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.968429 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.968560 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.968679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.968821 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.968940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.969058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] role.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.969175 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.969301 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] role.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.969420 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.969537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.969653 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.969771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.969889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.970081 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.970202 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.970320 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.970437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.970554 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.970670 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.970795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.970915 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.971032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.971149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.971265 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.971382 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.971507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.971625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.971742 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.971859 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.972032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.972152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] role.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.972282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.972402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] role.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.972520 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.972637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.972753 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.972879 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.972879 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.973101 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.973218 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.973334 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.973450 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.973574 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.973693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.973810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.973927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.974045 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.974162 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.974286 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.974405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.974522 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.974639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.974848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.974966 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.975094 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.975213 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.975330 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.975446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.975563 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.975681 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.975806 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.975806 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.976048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.976168 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.976286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.976926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.977057 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.977179 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.977297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.977414 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.977530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.977650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.977650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.977885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.978003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] token.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.978120 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.978238 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.978355 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.978481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.978600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.978717 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.978834 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.978951 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.979076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.979195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.979313 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.979430 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.979547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.979664 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.979790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.979909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.980048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.980167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.980284 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.980402 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.980531 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.980651 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.980768 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.980885 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.981003 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.981121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.981249 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.981369 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.981486 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.981603 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.981720 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.981852 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.982018 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.982139 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.982256 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.982373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.connection = **** {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.982500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.982628 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.982747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.982864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.982989 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.983107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.983230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.983348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.983466 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.983583 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.983711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.983830 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.983967 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.984087 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.984205 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.984322 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.984451 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.984572 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.984689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.984806 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.984923 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.985040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.985166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.985285 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.985403 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.985519 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.985519 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.985741 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.985873 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.986004 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.986123 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.986241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.986358 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.986475 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.986592 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.986718 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.986838 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80158) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 13:54:19.986955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.986955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.986955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.986955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.986955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.986955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.986955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.986955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.986955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.987916 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.987916 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.987916 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.987916 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.987916 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 13:54:19.987916 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 13:54:20.014879 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.015108 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.015329 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.017262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.017504 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.017735 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.024429 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.026693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.052019 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.052243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.052533 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.052770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.195428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.199430 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.244540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.244755 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.246689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.246928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.308762 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.308985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.309204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.309421 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.309523 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.309637 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.309721 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.309853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.309938 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.310151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.310375 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.310598 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.334559 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.334785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.335012 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.335099 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.335236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.335321 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.335454 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.335547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.335670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.336280 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.336447 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.336533 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.336677 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.336899 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.347456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.348027 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.360135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.360346 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.360568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.360651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.360795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.360879 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.361011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.361103 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.361223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.361306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.361435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80158) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.361518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.361703 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.361923 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 13:54:20.378831 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.379039 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 13:54:20.379373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.379532 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.379629 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 13:54:20.379746 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 13:54:20.379830 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.380104 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.380189 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.380310 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 13:54:20.380405 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.380584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.380667 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.380851 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.381114 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.381373 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.391642 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.391913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.392206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.392484 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.393447 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.393727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.394000 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.394286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.401317 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.402924 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.404568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.404845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.405124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.405515 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.405797 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.406057 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.406223 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.406330 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.406507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.406589 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.406795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.406878 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.407122 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.407203 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.407386 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.407469 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.407729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.408025 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.408304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.408580 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.408858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.409216 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.434445 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.436099 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.436380 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.436775 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.438430 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.438715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.441581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.441861 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.442139 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.443976 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.444262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.444545 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.449388 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.449619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.452095 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.452381 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.454451 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.454734 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.455015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.455294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.455565 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.455832 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.456130 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.456396 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.456991 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.457283 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.457568 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.457850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.458128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.458407 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.458683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.458955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.475294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.475567 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.478684 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.478960 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.480721 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.483484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.484485 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.484777 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.487126 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.487448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.489716 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.492483 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.493374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.496109 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.497115 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.497412 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.497703 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.497994 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.498277 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.498560 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.499842 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.500170 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.500459 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.500745 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.501045 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.501335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.517250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.517622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.517888 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.518153 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.520145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.520526 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.520800 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.521076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.526498 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.528235 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.528507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.529513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.531224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.531502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.531584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.531786 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.532094 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.532381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.532660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.534565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.534858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.535142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.535439 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.535732 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.546025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.546302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.546579 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.546852 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.547124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.547404 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.547674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.549167 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.549451 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.549736 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.550019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.550303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.550578 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.550866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.566871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.569973 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.570709 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.573804 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.574002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.575581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.577409 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.577512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.577772 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.578049 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.579029 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.580704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.580989 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.581275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.585370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.587006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.588759 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.590217 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.590372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.590504 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.590785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.591060 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.593697 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.593990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.594276 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.594555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.600205 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.600488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.600771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.604346 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.604636 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.604933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.609045 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80158) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.609323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.609600 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.609869 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.610144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.610426 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.610696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.611165 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.611265 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.611477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.611748 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.612039 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.612298 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.612556 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80158) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.612874 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 13:54:20.613155 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.613439 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.613720 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.614004 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.614293 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.614569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.614843 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.615120 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.615402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.615719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.615978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.616248 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.616515 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 13:54:20.651008 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55bbf06139c0 pid: 80158 (default app) Jul 18 13:54:20.654363 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a1d97-5fc0-4f37-a1be-eee5c7051b1d None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:20.654659 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a1d97-5fc0-4f37-a1be-eee5c7051b1d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:20.654918 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55bbf06139c0 pid: 80160 (default app) Jul 18 13:54:20.655002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a1d97-5fc0-4f37-a1be-eee5c7051b1d None None] PATH_INFO: `/v3` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:20.659103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1/1] 10.222.0.94 () {50 vars in 791 bytes} [Mon Jul 18 13:54:20 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:28.161644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d92b1-a312-4f41-aa77-bff5e1b1122d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:28.162198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d92b1-a312-4f41-aa77-bff5e1b1122d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:28.162198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d92b1-a312-4f41-aa77-bff5e1b1122d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:28.167055 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1/2] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:54:28.172440 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83644f1d-a356-460e-a86f-427ce399518d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:28.172871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83644f1d-a356-460e-a86f-427ce399518d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:28.173006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83644f1d-a356-460e-a86f-427ce399518d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:28.240820 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: INFO dbcounter [None req-83644f1d-a356-460e-a86f-427ce399518d None None] Registered counter for database keystone Jul 18 13:54:28.261929 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG oslo_db.sqlalchemy.engines [None req-83644f1d-a356-460e-a86f-427ce399518d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80158) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 13:54:28.268077 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writer thread running {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 13:54:28.441880 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG passlib.handlers.bcrypt [None req-83644f1d-a356-460e-a86f-427ce399518d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80158) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 13:54:28.442304 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG passlib.handlers.bcrypt [None req-83644f1d-a356-460e-a86f-427ce399518d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80158) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 13:54:28.497382 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-83644f1d-a356-460e-a86f-427ce399518d None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:28.498933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-83644f1d-a356-460e-a86f-427ce399518d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:28.621193 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 2/3] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 18 13:54:28.735785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d92b1-a312-4f41-aa77-bff5e1b1122d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:28.807857 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: INFO dbcounter [None req-3d1d92b1-a312-4f41-aa77-bff5e1b1122d None None] Registered counter for database keystone Jul 18 13:54:28.827063 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG oslo_db.sqlalchemy.engines [None req-3d1d92b1-a312-4f41-aa77-bff5e1b1122d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80160) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 13:54:28.833285 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writer thread running {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 13:54:28.952925 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d92b1-a312-4f41-aa77-bff5e1b1122d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:28.957287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56068708-46a0-4bdc-ad17-889bf7c9a594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:28.957955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56068708-46a0-4bdc-ad17-889bf7c9a594 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:28.958249 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56068708-46a0-4bdc-ad17-889bf7c9a594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:28.958536 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56068708-46a0-4bdc-ad17-889bf7c9a594 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:28.967483 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56068708-46a0-4bdc-ad17-889bf7c9a594 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:29.019179 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: WARNING keystone.common.rbac_enforcer.enforcer [None req-56068708-46a0-4bdc-ad17-889bf7c9a594 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 13:54:29.019559 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56068708-46a0-4bdc-ad17-889bf7c9a594 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:29.030025 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 2/4] 10.222.0.94 () {58 vars in 1324 bytes} [Mon Jul 18 13:54:28 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 13:54:29.036298 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83644f1d-a356-460e-a86f-427ce399518d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:29.050768 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83644f1d-a356-460e-a86f-427ce399518d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:29.055812 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c0052-70dc-4abb-bfe0-8dc934d8e6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:29.056970 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c0052-70dc-4abb-bfe0-8dc934d8e6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:29.057263 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c0052-70dc-4abb-bfe0-8dc934d8e6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:29.057544 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c0052-70dc-4abb-bfe0-8dc934d8e6d0 admin admin] PATH_INFO: `/v3/projects/defeadff0f9c417facade99f9905fca6` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:29.061696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c0052-70dc-4abb-bfe0-8dc934d8e6d0 admin admin] RBAC: Authorizing `identity:get_project(project_id=defeadff0f9c417facade99f9905fca6)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:29.113405 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fc3c0052-70dc-4abb-bfe0-8dc934d8e6d0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 13:54:29.113754 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c0052-70dc-4abb-bfe0-8dc934d8e6d0 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:29.118334 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 3/5] 10.222.0.94 () {56 vars in 1207 bytes} [Mon Jul 18 13:54:29 2022] GET /identity/v3/projects/defeadff0f9c417facade99f9905fca6 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:31.093517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a9d22-c711-47d4-9aa7-2a2c29e8ad25 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:31.094047 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a9d22-c711-47d4-9aa7-2a2c29e8ad25 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:31.094140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a9d22-c711-47d4-9aa7-2a2c29e8ad25 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:31.098204 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 3/6] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:54:31.103992 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531986a-938f-43c0-a44c-066e23257fab None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:31.104425 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531986a-938f-43c0-a44c-066e23257fab None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:31.104559 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531986a-938f-43c0-a44c-066e23257fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:31.123399 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-0531986a-938f-43c0-a44c-066e23257fab None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:31.124941 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0531986a-938f-43c0-a44c-066e23257fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:31.132762 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 4/7] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 18 13:54:31.153940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a9d22-c711-47d4-9aa7-2a2c29e8ad25 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:31.166499 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a9d22-c711-47d4-9aa7-2a2c29e8ad25 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:31.170719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c4d78-fb90-4079-a136-c68c625ca3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:31.171344 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c4d78-fb90-4079-a136-c68c625ca3d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:31.171637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c4d78-fb90-4079-a136-c68c625ca3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:31.171924 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c4d78-fb90-4079-a136-c68c625ca3d3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:31.179555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c4d78-fb90-4079-a136-c68c625ca3d3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:31.181479 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c4d78-fb90-4079-a136-c68c625ca3d3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:31.190167 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 4/8] 10.222.0.94 () {58 vars in 1324 bytes} [Mon Jul 18 13:54:31 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 13:54:31.195213 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531986a-938f-43c0-a44c-066e23257fab None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:31.204958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531986a-938f-43c0-a44c-066e23257fab None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:31.209263 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82035a5e-2576-4ccd-9002-b3a171aa26c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:31.210109 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82035a5e-2576-4ccd-9002-b3a171aa26c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:31.210397 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82035a5e-2576-4ccd-9002-b3a171aa26c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:31.210675 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82035a5e-2576-4ccd-9002-b3a171aa26c9 admin admin] PATH_INFO: `/v3/users/7a10bd46534e415080cc3fb7cb29bb4d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:31.212661 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82035a5e-2576-4ccd-9002-b3a171aa26c9 admin admin] RBAC: Authorizing `identity:get_user(user_id=7a10bd46534e415080cc3fb7cb29bb4d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:31.214486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82035a5e-2576-4ccd-9002-b3a171aa26c9 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:31.216033 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 5/9] 10.222.0.94 () {56 vars in 1198 bytes} [Mon Jul 18 13:54:31 2022] GET /identity/v3/users/7a10bd46534e415080cc3fb7cb29bb4d => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:33.236570 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb060cf-4efa-43b6-af25-321d85c32fe2 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:33.237087 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb060cf-4efa-43b6-af25-321d85c32fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:33.237087 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb060cf-4efa-43b6-af25-321d85c32fe2 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:33.238398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 5/10] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:33.243921 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5ede2-de24-408c-9d1d-8b931061f382 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:33.244337 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5ede2-de24-408c-9d1d-8b931061f382 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:33.244512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5ede2-de24-408c-9d1d-8b931061f382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:33.260883 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d5a5ede2-de24-408c-9d1d-8b931061f382 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:33.262049 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d5a5ede2-de24-408c-9d1d-8b931061f382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:33.306394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c050ef-101a-4257-9851-07027446be69 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:33.306686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c050ef-101a-4257-9851-07027446be69 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:33.306973 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c050ef-101a-4257-9851-07027446be69 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:33.308373 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 6/11] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:54:33.313341 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89881c2b-fac4-450b-a633-269101cb2210 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:33.313642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89881c2b-fac4-450b-a633-269101cb2210 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:33.314203 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89881c2b-fac4-450b-a633-269101cb2210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:33.341185 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 6/12] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:54:33.354664 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG passlib.handlers.bcrypt [None req-89881c2b-fac4-450b-a633-269101cb2210 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80160) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 13:54:33.355096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG passlib.handlers.bcrypt [None req-89881c2b-fac4-450b-a633-269101cb2210 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80160) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 13:54:33.364069 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a5ede2-de24-408c-9d1d-8b931061f382 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:33.387209 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:33.402368 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-89881c2b-fac4-450b-a633-269101cb2210 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:33.404004 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-89881c2b-fac4-450b-a633-269101cb2210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:33.465764 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a5ede2-de24-408c-9d1d-8b931061f382 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:33.498555 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd5164-651f-4d8c-a1a4-d4eac12ae929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:33.499285 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd5164-651f-4d8c-a1a4-d4eac12ae929 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:33.499580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd5164-651f-4d8c-a1a4-d4eac12ae929 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:33.499857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd5164-651f-4d8c-a1a4-d4eac12ae929 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:33.503023 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 7/13] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 13:54:33.512784 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd5164-651f-4d8c-a1a4-d4eac12ae929 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:54:33.523881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-e8cd5164-651f-4d8c-a1a4-d4eac12ae929 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:54:33.523881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:54:33.523881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:54:33.523881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:54:33.523881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:54:33.523881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:54:33.523881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:54:33.523881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:54:33.523881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:54:33.523881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:54:33.523881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:54:33.523881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:54:33.523881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:54:33.523881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:54:33.523881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:54:33.523881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:54:33.525517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:54:33.525517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:54:33.525517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:54:33.525517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:54:33.525517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:54:33.525517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:54:33.525517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:54:33.525517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:54:33.525517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:54:33.525517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:54:33.525517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:54:33.525517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:54:33.525517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:54:33.525517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:54:33.525517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:54:33.525517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:54:33.526537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:54:33.526537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:54:33.526537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:54:33.526537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:54:33.526537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:54:33.526537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:54:33.526537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:54:33.526537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:54:33.526537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:54:33.526537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:54:33.526537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:54:33.526537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:54:33.527298 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89881c2b-fac4-450b-a633-269101cb2210 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:33.527385 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 7/14] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:54:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:54:33.533134 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd5164-651f-4d8c-a1a4-d4eac12ae929 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:33.612696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:33.624638 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89881c2b-fac4-450b-a633-269101cb2210 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:33.626965 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:33.631906 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd5164-651f-4d8c-a1a4-d4eac12ae929 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:33.664203 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b55d34-0695-4682-9db5-ea0858868526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:33.665109 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b55d34-0695-4682-9db5-ea0858868526 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:33.665401 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b55d34-0695-4682-9db5-ea0858868526 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:33.665686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b55d34-0695-4682-9db5-ea0858868526 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:33.667422 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c8866-b413-4fd4-837d-5b513d558bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:33.668132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c8866-b413-4fd4-837d-5b513d558bbc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:33.668428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c8866-b413-4fd4-837d-5b513d558bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:33.668710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c8866-b413-4fd4-837d-5b513d558bbc None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:33.669465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c8866-b413-4fd4-837d-5b513d558bbc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:33.669783 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c8866-b413-4fd4-837d-5b513d558bbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:33.671203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c8866-b413-4fd4-837d-5b513d558bbc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:33.683425 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 8/15] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:54:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:54:33.689125 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c8866-b413-4fd4-837d-5b513d558bbc None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:33.733229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b55d34-0695-4682-9db5-ea0858868526 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:33.734848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b55d34-0695-4682-9db5-ea0858868526 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:33.788345 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c8866-b413-4fd4-837d-5b513d558bbc None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:33.822612 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f908749-eafc-417b-b055-f018707cdab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:33.823480 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f908749-eafc-417b-b055-f018707cdab9 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:33.823772 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f908749-eafc-417b-b055-f018707cdab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:33.824089 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f908749-eafc-417b-b055-f018707cdab9 None admin] PATH_INFO: `/v3/users/7a10bd46534e415080cc3fb7cb29bb4d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:33.825306 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f908749-eafc-417b-b055-f018707cdab9 None admin] RBAC: Authorizing `identity:get_user(user_id=7a10bd46534e415080cc3fb7cb29bb4d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:33.826775 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f908749-eafc-417b-b055-f018707cdab9 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:33.828703 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 9/16] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:54:33 2022] GET /identity/v3/users/7a10bd46534e415080cc3fb7cb29bb4d => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:54:33.834596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 8/17] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:54:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:54:33.834718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f908749-eafc-417b-b055-f018707cdab9 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:33.835495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b55d34-0695-4682-9db5-ea0858868526 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:33.838877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:33.850568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:33.932335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b55d34-0695-4682-9db5-ea0858868526 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:33.935033 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f908749-eafc-417b-b055-f018707cdab9 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:33.966656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a380a0-acea-435a-8888-fc996d823992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:33.967454 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a380a0-acea-435a-8888-fc996d823992 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:33.967744 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a380a0-acea-435a-8888-fc996d823992 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:33.968066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a380a0-acea-435a-8888-fc996d823992 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:33.969686 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282545b-d93f-4edc-ac60-f300cd0a4df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:33.970490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282545b-d93f-4edc-ac60-f300cd0a4df0 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:33.970776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282545b-d93f-4edc-ac60-f300cd0a4df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:33.971058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282545b-d93f-4edc-ac60-f300cd0a4df0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:33.972283 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282545b-d93f-4edc-ac60-f300cd0a4df0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:33.973744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282545b-d93f-4edc-ac60-f300cd0a4df0 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:33.975341 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 10/18] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:54:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:33.981043 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282545b-d93f-4edc-ac60-f300cd0a4df0 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:33.988233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a380a0-acea-435a-8888-fc996d823992 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:33.989720 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a380a0-acea-435a-8888-fc996d823992 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:34.010625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-04a380a0-acea-435a-8888-fc996d823992 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:54:34.010625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:54:34.010625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:54:34.010625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:54:34.010625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:54:34.010625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:54:34.010625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:54:34.010625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:54:34.010625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:54:34.010625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:54:34.010625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:54:34.010625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:54:34.010625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:54:34.010625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:54:34.010625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:54:34.011485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:54:34.011485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:54:34.011485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:54:34.011485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:54:34.011485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:54:34.011485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:54:34.011485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:54:34.011485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:54:34.011485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:54:34.011485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:54:34.011485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:54:34.011485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:54:34.011485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:54:34.011485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:54:34.011485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:54:34.011485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:54:34.011485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:54:34.012329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:54:34.012329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:54:34.012329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:54:34.012329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:54:34.012329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 9/19] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:54:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:54:34.017474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a380a0-acea-435a-8888-fc996d823992 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:34.062617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:34.073340 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:34.084462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282545b-d93f-4edc-ac60-f300cd0a4df0 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:34.117445 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a380a0-acea-435a-8888-fc996d823992 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:34.119269 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8749da-3094-4141-a280-a4af5b5eab87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:34.119930 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8749da-3094-4141-a280-a4af5b5eab87 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:34.120251 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8749da-3094-4141-a280-a4af5b5eab87 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:34.120537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8749da-3094-4141-a280-a4af5b5eab87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:34.121363 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8749da-3094-4141-a280-a4af5b5eab87 None admin] RBAC: Adding query filter params (role.id=c60a99de086f45c6820f7872a6b43feb, scope.domain.id=default, user.id=7a10bd46534e415080cc3fb7cb29bb4d) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:34.121678 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8749da-3094-4141-a280-a4af5b5eab87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:34.123117 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8749da-3094-4141-a280-a4af5b5eab87 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:34.129913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 11/20] 10.222.0.94 () {56 vars in 1322 bytes} [Mon Jul 18 13:54:33 2022] GET /identity/v3/role_assignments?user.id=7a10bd46534e415080cc3fb7cb29bb4d&scope.domain.id=default&role.id=c60a99de086f45c6820f7872a6b43feb => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:54:34.152851 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2601932-302c-4b7b-bb7d-085ead893cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:34.153549 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2601932-302c-4b7b-bb7d-085ead893cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:34.153845 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2601932-302c-4b7b-bb7d-085ead893cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:34.154130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2601932-302c-4b7b-bb7d-085ead893cf7 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:34.154888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2601932-302c-4b7b-bb7d-085ead893cf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:34.155204 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2601932-302c-4b7b-bb7d-085ead893cf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:34.156714 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2601932-302c-4b7b-bb7d-085ead893cf7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:34.175891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 10/21] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:54:36.469363 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508586f7-07de-46c2-b665-37257bc8a7bb None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:36.469888 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508586f7-07de-46c2-b665-37257bc8a7bb None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:36.469970 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508586f7-07de-46c2-b665-37257bc8a7bb None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:36.471586 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 12/22] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:36.477932 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73532de8-6890-418b-9d83-0247af40cf2b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:36.478398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73532de8-6890-418b-9d83-0247af40cf2b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:36.478598 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73532de8-6890-418b-9d83-0247af40cf2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:36.505126 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-73532de8-6890-418b-9d83-0247af40cf2b None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:36.506349 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-73532de8-6890-418b-9d83-0247af40cf2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:36.548997 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d42c5-63ab-485e-92c8-0b03c2a5d8fc None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:36.549292 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d42c5-63ab-485e-92c8-0b03c2a5d8fc None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:36.549563 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d42c5-63ab-485e-92c8-0b03c2a5d8fc None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:36.550934 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 13/23] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:36.556300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3f897-1362-4898-9332-f977d48a63ec None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:36.556599 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3f897-1362-4898-9332-f977d48a63ec None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:36.556889 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3f897-1362-4898-9332-f977d48a63ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:36.592131 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-fed3f897-1362-4898-9332-f977d48a63ec None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:36.593356 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fed3f897-1362-4898-9332-f977d48a63ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:36.627760 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 11/24] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 13:54:36.659378 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73532de8-6890-418b-9d83-0247af40cf2b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:36.726371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 14/25] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:54:36.754717 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:36.780727 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3f897-1362-4898-9332-f977d48a63ec None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:36.841760 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73532de8-6890-418b-9d83-0247af40cf2b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:36.893966 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a1790-1bb3-495d-9bcf-2800eb64ab26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:36.894784 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a1790-1bb3-495d-9bcf-2800eb64ab26 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:36.895073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a1790-1bb3-495d-9bcf-2800eb64ab26 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:36.895354 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a1790-1bb3-495d-9bcf-2800eb64ab26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:36.918883 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a1790-1bb3-495d-9bcf-2800eb64ab26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:54:36.931335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b76a1790-1bb3-495d-9bcf-2800eb64ab26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:54:36.931335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:54:36.931335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:54:36.931335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:54:36.931335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:54:36.931335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:54:36.931335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:54:36.931335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:54:36.931335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:54:36.931335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:54:36.931335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:54:36.931335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:54:36.931335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:54:36.931335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:54:36.931335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:54:36.931335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:54:36.932303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:54:36.932303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:54:36.932303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:54:36.932303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:54:36.932303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:54:36.932303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:54:36.932303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:54:36.932303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:54:36.932303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:54:36.932303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:54:36.932303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:54:36.932303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:54:36.932303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:54:36.932303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:54:36.932303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:54:36.932303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:54:36.933060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:54:36.933060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:54:36.933060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:54:36.933060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:54:36.933060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:54:36.933060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:54:36.933060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:54:36.933060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:54:36.933060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:54:36.933060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:54:36.933060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:54:36.933060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:54:36.933060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 12/26] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:54:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:54:36.934511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa21a99-2d2a-4ff0-bb20-e20473266583 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:36.934511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa21a99-2d2a-4ff0-bb20-e20473266583 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:36.934812 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa21a99-2d2a-4ff0-bb20-e20473266583 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:36.939645 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 13/27] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:36.943705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa21a99-2d2a-4ff0-bb20-e20473266583 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:36.980811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:37.010219 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3f897-1362-4898-9332-f977d48a63ec None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:37.049002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34a8cf-0def-4d58-aab0-674a1c129796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:37.050159 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34a8cf-0def-4d58-aab0-674a1c129796 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:37.050642 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34a8cf-0def-4d58-aab0-674a1c129796 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:37.051091 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34a8cf-0def-4d58-aab0-674a1c129796 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:37.062324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e34a8cf-0def-4d58-aab0-674a1c129796 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:37.064156 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e34a8cf-0def-4d58-aab0-674a1c129796 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:37.085348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-8e34a8cf-0def-4d58-aab0-674a1c129796 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:54:37.085348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:54:37.085348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:54:37.085348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:54:37.085348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:54:37.085348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:54:37.085348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:54:37.085348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:54:37.085348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:54:37.085348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:54:37.085348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:54:37.085348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:54:37.085348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:54:37.085348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:54:37.085348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:54:37.086486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:54:37.086486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:54:37.086486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:54:37.086486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:54:37.086486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:54:37.086486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:54:37.086486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:54:37.086486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:54:37.086486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:54:37.086486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:54:37.086486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:54:37.086486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:54:37.086486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:54:37.086486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:54:37.086486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:54:37.086486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:54:37.086486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:54:37.087371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:54:37.087371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:54:37.087371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:54:37.087371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:54:37.089329 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 15/28] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:54:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:54:37.090719 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d54d8-2d5a-4261-b4a1-6151234c2743 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:37.091160 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d54d8-2d5a-4261-b4a1-6151234c2743 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:37.091584 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d54d8-2d5a-4261-b4a1-6151234c2743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:37.097623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa21a99-2d2a-4ff0-bb20-e20473266583 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:37.106103 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:37.111027 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-8a5d54d8-2d5a-4261-b4a1-6151234c2743 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:37.112589 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8a5d54d8-2d5a-4261-b4a1-6151234c2743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:37.147461 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a3ffe-9eb8-4121-9ef0-158af8dfec28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:37.148214 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a3ffe-9eb8-4121-9ef0-158af8dfec28 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:37.148506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a3ffe-9eb8-4121-9ef0-158af8dfec28 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:37.148819 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6a3ffe-9eb8-4121-9ef0-158af8dfec28 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:37.149650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a3ffe-9eb8-4121-9ef0-158af8dfec28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:37.149966 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a3ffe-9eb8-4121-9ef0-158af8dfec28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:37.151433 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6a3ffe-9eb8-4121-9ef0-158af8dfec28 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:37.164066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 14/29] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:54:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:37.165665 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a3ffe-9eb8-4121-9ef0-158af8dfec28 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:37.190856 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 16/30] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:54:37.192551 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9e21b-4158-40a7-9dc4-7cd8fcbaabf5 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:37.192992 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9e21b-4158-40a7-9dc4-7cd8fcbaabf5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:37.193442 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9e21b-4158-40a7-9dc4-7cd8fcbaabf5 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:37.194991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 17/31] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:37.196555 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9e21b-4158-40a7-9dc4-7cd8fcbaabf5 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:37.275066 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:37.284182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6a3ffe-9eb8-4121-9ef0-158af8dfec28 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:37.331405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34409c23-04ed-4cba-a335-56f7e070b104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:37.332291 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34409c23-04ed-4cba-a335-56f7e070b104 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:37.332657 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34409c23-04ed-4cba-a335-56f7e070b104 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:37.333017 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34409c23-04ed-4cba-a335-56f7e070b104 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:37.333916 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34409c23-04ed-4cba-a335-56f7e070b104 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:37.334372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34409c23-04ed-4cba-a335-56f7e070b104 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:37.335805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34409c23-04ed-4cba-a335-56f7e070b104 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:37.339276 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9e21b-4158-40a7-9dc4-7cd8fcbaabf5 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:37.349089 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 15/32] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:37.355086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39297f-8bcd-4238-93f1-0f4ce0b46f4e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:37.355457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39297f-8bcd-4238-93f1-0f4ce0b46f4e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:37.355822 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39297f-8bcd-4238-93f1-0f4ce0b46f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:37.373934 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1f39297f-8bcd-4238-93f1-0f4ce0b46f4e None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:37.375349 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1f39297f-8bcd-4238-93f1-0f4ce0b46f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:37.381877 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09cad09-3c42-4d48-99e4-da16fdb28a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:37.383020 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cad09-3c42-4d48-99e4-da16fdb28a80 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:37.384028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cad09-3c42-4d48-99e4-da16fdb28a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:37.385011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cad09-3c42-4d48-99e4-da16fdb28a80 None admin] PATH_INFO: `/v3/users/7a10bd46534e415080cc3fb7cb29bb4d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:37.386414 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09cad09-3c42-4d48-99e4-da16fdb28a80 None admin] RBAC: Authorizing `identity:get_user(user_id=7a10bd46534e415080cc3fb7cb29bb4d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:37.389013 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09cad09-3c42-4d48-99e4-da16fdb28a80 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:37.391150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 18/33] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:54:37 2022] GET /identity/v3/users/7a10bd46534e415080cc3fb7cb29bb4d => generated 247 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:37.392094 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09cad09-3c42-4d48-99e4-da16fdb28a80 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:37.408563 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:37.461876 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 16/34] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:54:37.463105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39297f-8bcd-4238-93f1-0f4ce0b46f4e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:37.535254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09cad09-3c42-4d48-99e4-da16fdb28a80 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:37.568140 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:37.575830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39297f-8bcd-4238-93f1-0f4ce0b46f4e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:37.583154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e3ce6-2d21-44d3-82e1-1949e7d69623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:37.584114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e3ce6-2d21-44d3-82e1-1949e7d69623 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:37.585008 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e3ce6-2d21-44d3-82e1-1949e7d69623 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:37.585494 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e3ce6-2d21-44d3-82e1-1949e7d69623 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:37.586981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e3ce6-2d21-44d3-82e1-1949e7d69623 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:37.589228 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e3ce6-2d21-44d3-82e1-1949e7d69623 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:37.617267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3f772-64a5-40fb-b499-3a8e96225ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:37.618549 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3f772-64a5-40fb-b499-3a8e96225ac1 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:37.618915 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3f772-64a5-40fb-b499-3a8e96225ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:37.619276 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3f772-64a5-40fb-b499-3a8e96225ac1 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:37.620436 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d3f772-64a5-40fb-b499-3a8e96225ac1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:37.622020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d3f772-64a5-40fb-b499-3a8e96225ac1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:37.721868 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 19/35] 10.222.0.94 () {60 vars in 1131 bytes} [Mon Jul 18 13:54:37 2022] POST /identity/v3/roles => generated 219 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:54:37.723224 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e3ce6-2d21-44d3-82e1-1949e7d69623 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:37.816361 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 17/36] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Jul 18 13:54:37 2022] POST /identity/v3/projects => generated 301 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:54:37.817450 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3f772-64a5-40fb-b499-3a8e96225ac1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:37.837395 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e3ce6-2d21-44d3-82e1-1949e7d69623 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:37.850613 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:37.874467 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b255196-d3fd-4e66-b4c6-90ae0df2b958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:37.875680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b255196-d3fd-4e66-b4c6-90ae0df2b958 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:37.876215 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b255196-d3fd-4e66-b4c6-90ae0df2b958 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:37.876722 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b255196-d3fd-4e66-b4c6-90ae0df2b958 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:37.878398 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b255196-d3fd-4e66-b4c6-90ae0df2b958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:37.880318 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b255196-d3fd-4e66-b4c6-90ae0df2b958 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:37.882463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 20/37] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:54:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:37.910539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b255196-d3fd-4e66-b4c6-90ae0df2b958 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:37.973229 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:38.026507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3f772-64a5-40fb-b499-3a8e96225ac1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:38.031838 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b255196-d3fd-4e66-b4c6-90ae0df2b958 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:38.068701 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18206d-bbb6-4eb8-8c37-f2093f4a0274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:38.069873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18206d-bbb6-4eb8-8c37-f2093f4a0274 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:38.070373 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18206d-bbb6-4eb8-8c37-f2093f4a0274 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:38.071429 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18206d-bbb6-4eb8-8c37-f2093f4a0274 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:38.073409 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d0edd-d7c7-4bb7-9ed6-84148d8740da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:38.074288 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18206d-bbb6-4eb8-8c37-f2093f4a0274 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:38.074775 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d0edd-d7c7-4bb7-9ed6-84148d8740da None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:38.075208 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d0edd-d7c7-4bb7-9ed6-84148d8740da None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:38.075674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d0edd-d7c7-4bb7-9ed6-84148d8740da None admin] PATH_INFO: `/v3/domains/default/users/7a10bd46534e415080cc3fb7cb29bb4d/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:38.078559 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d0edd-d7c7-4bb7-9ed6-84148d8740da None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7a10bd46534e415080cc3fb7cb29bb4d, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:38.079073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18206d-bbb6-4eb8-8c37-f2093f4a0274 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:38.080613 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d0edd-d7c7-4bb7-9ed6-84148d8740da None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:38.098578 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 21/38] 10.222.0.94 () {58 vars in 1361 bytes} [Mon Jul 18 13:54:37 2022] PUT /identity/v3/domains/default/users/7a10bd46534e415080cc3fb7cb29bb4d/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:54:38.100233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 18/39] 10.222.0.94 () {60 vars in 1131 bytes} [Mon Jul 18 13:54:37 2022] POST /identity/v3/roles => generated 225 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:54:38.100742 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20236d15-71dd-4a35-be0b-6343e0a0ab8a None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:38.101734 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20236d15-71dd-4a35-be0b-6343e0a0ab8a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:38.101734 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20236d15-71dd-4a35-be0b-6343e0a0ab8a None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:38.101949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d13950-b326-4e85-9bdc-424a547065d7 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:38.102092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d13950-b326-4e85-9bdc-424a547065d7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:38.102347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d13950-b326-4e85-9bdc-424a547065d7 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:38.103861 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 22/40] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:38.109320 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 19/41] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:38.111647 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af6070-d3c9-4385-8cc2-10f16550bb48 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:38.111843 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88d415-700a-48fd-bc97-c46439b105a6 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:38.112024 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af6070-d3c9-4385-8cc2-10f16550bb48 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:38.112296 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af6070-d3c9-4385-8cc2-10f16550bb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:38.112666 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88d415-700a-48fd-bc97-c46439b105a6 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:38.113122 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88d415-700a-48fd-bc97-c46439b105a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:38.140125 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-9b88d415-700a-48fd-bc97-c46439b105a6 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:38.141795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9b88d415-700a-48fd-bc97-c46439b105a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:38.144823 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-70af6070-d3c9-4385-8cc2-10f16550bb48 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:38.147758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-70af6070-d3c9-4385-8cc2-10f16550bb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:38.252748 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 23/42] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:54:38.255038 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:38.262619 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 20/43] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 13:54:38.279444 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b88d415-700a-48fd-bc97-c46439b105a6 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:38.286138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af6070-d3c9-4385-8cc2-10f16550bb48 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:38.421118 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af6070-d3c9-4385-8cc2-10f16550bb48 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:38.449495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:38.487081 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c596e8-1f6c-4e4c-ba53-8fc4349fef9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:38.487966 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c596e8-1f6c-4e4c-ba53-8fc4349fef9f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:38.488310 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c596e8-1f6c-4e4c-ba53-8fc4349fef9f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:38.488597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c596e8-1f6c-4e4c-ba53-8fc4349fef9f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:38.489790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c596e8-1f6c-4e4c-ba53-8fc4349fef9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:38.491291 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c596e8-1f6c-4e4c-ba53-8fc4349fef9f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:38.496424 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 21/44] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:54:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:54:38.498353 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c596e8-1f6c-4e4c-ba53-8fc4349fef9f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:38.517272 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b88d415-700a-48fd-bc97-c46439b105a6 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:38.553078 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdfea05-aa21-4a9e-b95b-4e1212ab288a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:38.553874 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdfea05-aa21-4a9e-b95b-4e1212ab288a None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:38.554177 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdfea05-aa21-4a9e-b95b-4e1212ab288a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:38.554465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdfea05-aa21-4a9e-b95b-4e1212ab288a None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:38.555796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdfea05-aa21-4a9e-b95b-4e1212ab288a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:38.557073 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdfea05-aa21-4a9e-b95b-4e1212ab288a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:38.570527 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 24/45] 10.222.0.94 () {60 vars in 1131 bytes} [Mon Jul 18 13:54:38 2022] POST /identity/v3/roles => generated 223 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:54:38.580000 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:38.615877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c596e8-1f6c-4e4c-ba53-8fc4349fef9f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:38.656050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e249fc-1055-449e-88e7-1a3494437c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:38.656747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e249fc-1055-449e-88e7-1a3494437c65 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:38.657046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e249fc-1055-449e-88e7-1a3494437c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:38.657337 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e249fc-1055-449e-88e7-1a3494437c65 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:38.658089 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e249fc-1055-449e-88e7-1a3494437c65 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:38.659596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e249fc-1055-449e-88e7-1a3494437c65 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:38.687435 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 22/46] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Jul 18 13:54:38 2022] POST /identity/v3/projects => generated 312 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 13:54:40.175805 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c2038-66b6-4ad0-b3a3-be527aba0ed8 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:40.176348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c2038-66b6-4ad0-b3a3-be527aba0ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:40.176430 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c2038-66b6-4ad0-b3a3-be527aba0ed8 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:40.177777 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 25/47] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:40.183478 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de907b67-6631-4410-b447-73fa404f1f78 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:40.183889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de907b67-6631-4410-b447-73fa404f1f78 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:40.184111 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de907b67-6631-4410-b447-73fa404f1f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:40.200604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-de907b67-6631-4410-b447-73fa404f1f78 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:40.201762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-de907b67-6631-4410-b447-73fa404f1f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:40.267543 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 23/48] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:54:40.290264 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c2038-66b6-4ad0-b3a3-be527aba0ed8 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:40.389657 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c2038-66b6-4ad0-b3a3-be527aba0ed8 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:40.391176 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:40.425256 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff08d3-7bc3-45e1-b54a-ac03eecdc53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:40.426059 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff08d3-7bc3-45e1-b54a-ac03eecdc53a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:40.426354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff08d3-7bc3-45e1-b54a-ac03eecdc53a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:40.426639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff08d3-7bc3-45e1-b54a-ac03eecdc53a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:40.437546 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff08d3-7bc3-45e1-b54a-ac03eecdc53a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:54:40.445189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-f5ff08d3-7bc3-45e1-b54a-ac03eecdc53a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:54:40.445189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:54:40.445189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:54:40.445189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:54:40.445189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:54:40.445189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:54:40.445189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:54:40.445189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:54:40.445189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:54:40.445189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:54:40.445189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:54:40.445189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:54:40.445189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:54:40.445189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:54:40.445189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:54:40.445189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:54:40.446688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:54:40.446688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:54:40.446688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:54:40.446688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:54:40.446688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:54:40.446688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:54:40.446688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:54:40.446688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:54:40.446688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:54:40.446688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:54:40.446688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:54:40.446688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:54:40.446688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:54:40.446688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:54:40.446688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:54:40.446688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:54:40.447514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:54:40.447514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:54:40.447514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:54:40.447514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:54:40.447514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:54:40.447514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:54:40.447514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:54:40.447514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:54:40.447514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:54:40.447514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:54:40.447514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:54:40.447514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:54:40.447514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 26/49] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:54:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:54:40.453233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de907b67-6631-4410-b447-73fa404f1f78 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:40.501382 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:40.550947 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de907b67-6631-4410-b447-73fa404f1f78 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:40.584747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde817d-84ed-4de8-bc10-aa84e3844b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:40.585424 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde817d-84ed-4de8-bc10-aa84e3844b40 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:40.585719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde817d-84ed-4de8-bc10-aa84e3844b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:40.586006 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde817d-84ed-4de8-bc10-aa84e3844b40 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:40.586900 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde817d-84ed-4de8-bc10-aa84e3844b40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:40.587216 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde817d-84ed-4de8-bc10-aa84e3844b40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:40.588656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde817d-84ed-4de8-bc10-aa84e3844b40 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:40.596754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 24/50] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:54:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:40.602238 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff08d3-7bc3-45e1-b54a-ac03eecdc53a None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:40.675558 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5673a57-fa7b-43e6-b700-72f83560602e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:40.675850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5673a57-fa7b-43e6-b700-72f83560602e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:40.676166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5673a57-fa7b-43e6-b700-72f83560602e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:40.677535 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 25/51] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:54:40.683395 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bface3f-e533-47c3-9d2a-c76256c08c24 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:40.683688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bface3f-e533-47c3-9d2a-c76256c08c24 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:40.683991 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bface3f-e533-47c3-9d2a-c76256c08c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:40.701029 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4bface3f-e533-47c3-9d2a-c76256c08c24 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:40.702237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4bface3f-e533-47c3-9d2a-c76256c08c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:40.704432 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff08d3-7bc3-45e1-b54a-ac03eecdc53a None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:40.738846 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c0b5a-1a4a-4a05-bb1a-32ec48a55976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:40.739718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c0b5a-1a4a-4a05-bb1a-32ec48a55976 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:40.740030 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c0b5a-1a4a-4a05-bb1a-32ec48a55976 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:40.740313 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c0b5a-1a4a-4a05-bb1a-32ec48a55976 None admin] PATH_INFO: `/v3/users/7a10bd46534e415080cc3fb7cb29bb4d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:40.741515 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c0b5a-1a4a-4a05-bb1a-32ec48a55976 None admin] RBAC: Authorizing `identity:get_user(user_id=7a10bd46534e415080cc3fb7cb29bb4d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:40.743005 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c0b5a-1a4a-4a05-bb1a-32ec48a55976 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:40.744681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 27/52] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:54:40 2022] GET /identity/v3/users/7a10bd46534e415080cc3fb7cb29bb4d => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:40.750351 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c0b5a-1a4a-4a05-bb1a-32ec48a55976 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:40.782907 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:40.856072 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c0b5a-1a4a-4a05-bb1a-32ec48a55976 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:40.875116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 26/53] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 13:54:40.889626 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05180152-d393-4057-b758-98436f73fc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:40.890422 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05180152-d393-4057-b758-98436f73fc5c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:40.890710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05180152-d393-4057-b758-98436f73fc5c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:40.890991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05180152-d393-4057-b758-98436f73fc5c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:40.892196 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05180152-d393-4057-b758-98436f73fc5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:40.893733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05180152-d393-4057-b758-98436f73fc5c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:40.895466 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 28/54] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:54:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:54:40.898020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bface3f-e533-47c3-9d2a-c76256c08c24 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:40.901733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05180152-d393-4057-b758-98436f73fc5c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:40.941679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:40.998484 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:40.999615 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bface3f-e533-47c3-9d2a-c76256c08c24 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:41.002523 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05180152-d393-4057-b758-98436f73fc5c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:41.033575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d7af2-e204-48d1-947d-a24e8e475103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:41.034376 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d7af2-e204-48d1-947d-a24e8e475103 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:41.034727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d7af2-e204-48d1-947d-a24e8e475103 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:41.034955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d7af2-e204-48d1-947d-a24e8e475103 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:41.036170 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d7af2-e204-48d1-947d-a24e8e475103 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:41.037646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d7af2-e204-48d1-947d-a24e8e475103 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:41.038035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9ec5b-d50c-47a6-aecd-5cfe0522615a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:41.038693 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9ec5b-d50c-47a6-aecd-5cfe0522615a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:41.038984 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9ec5b-d50c-47a6-aecd-5cfe0522615a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:41.039444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 27/55] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:54:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:41.039543 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9ec5b-d50c-47a6-aecd-5cfe0522615a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:41.040155 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa9ec5b-d50c-47a6-aecd-5cfe0522615a None admin] RBAC: Adding query filter params (role.id=c60a99de086f45c6820f7872a6b43feb, scope.domain.id=default, user.id=7a10bd46534e415080cc3fb7cb29bb4d) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:41.040473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa9ec5b-d50c-47a6-aecd-5cfe0522615a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:41.041903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa9ec5b-d50c-47a6-aecd-5cfe0522615a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:41.044719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d7af2-e204-48d1-947d-a24e8e475103 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:41.047806 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 29/56] 10.222.0.94 () {56 vars in 1322 bytes} [Mon Jul 18 13:54:40 2022] GET /identity/v3/role_assignments?user.id=7a10bd46534e415080cc3fb7cb29bb4d&scope.domain.id=default&role.id=c60a99de086f45c6820f7872a6b43feb => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:54:41.154194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d7af2-e204-48d1-947d-a24e8e475103 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:41.165462 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:41.190662 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140b2fd-6d91-40c8-b122-56a5ff799ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:41.191350 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140b2fd-6d91-40c8-b122-56a5ff799ac8 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:41.191613 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140b2fd-6d91-40c8-b122-56a5ff799ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:41.191891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140b2fd-6d91-40c8-b122-56a5ff799ac8 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:41.192654 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140b2fd-6d91-40c8-b122-56a5ff799ac8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:41.194127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140b2fd-6d91-40c8-b122-56a5ff799ac8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:41.220320 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 28/57] 10.222.0.94 () {60 vars in 1140 bytes} [Mon Jul 18 13:54:41 2022] POST /identity/v3/projects => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 13:54:43.209532 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc990a-f2a7-488d-9408-7f536241462d None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:43.210009 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc990a-f2a7-488d-9408-7f536241462d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:43.210009 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc990a-f2a7-488d-9408-7f536241462d None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:43.211407 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 30/58] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:43.217001 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1084dec-7cb1-48fa-ad1d-bc40bcdce152 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:43.217357 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1084dec-7cb1-48fa-ad1d-bc40bcdce152 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:43.217571 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1084dec-7cb1-48fa-ad1d-bc40bcdce152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:43.233796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c1084dec-7cb1-48fa-ad1d-bc40bcdce152 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:43.234995 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c1084dec-7cb1-48fa-ad1d-bc40bcdce152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:43.298668 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 29/59] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:54:43.322267 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc990a-f2a7-488d-9408-7f536241462d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:43.420484 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc990a-f2a7-488d-9408-7f536241462d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:43.453027 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d0add-3e4d-4f75-b4b0-82009fa6a0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:43.453807 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d0add-3e4d-4f75-b4b0-82009fa6a0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:43.454099 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d0add-3e4d-4f75-b4b0-82009fa6a0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:43.454385 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d0add-3e4d-4f75-b4b0-82009fa6a0c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:43.455553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d0add-3e4d-4f75-b4b0-82009fa6a0c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:43.457044 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d0add-3e4d-4f75-b4b0-82009fa6a0c8 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:43.458636 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 31/60] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:54:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:43.464537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1084dec-7cb1-48fa-ad1d-bc40bcdce152 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:43.551814 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:43.561538 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1084dec-7cb1-48fa-ad1d-bc40bcdce152 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:43.594762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c76de-06db-48a7-a7fe-ef4cf199d743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:43.595441 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c76de-06db-48a7-a7fe-ef4cf199d743 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:43.595735 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c76de-06db-48a7-a7fe-ef4cf199d743 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:43.596057 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c76de-06db-48a7-a7fe-ef4cf199d743 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:43.596783 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c76de-06db-48a7-a7fe-ef4cf199d743 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:43.598241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c76de-06db-48a7-a7fe-ef4cf199d743 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:43.625428 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 30/61] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Jul 18 13:54:43 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 13:54:46.794868 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42913b-3cf0-4592-a080-12cccc4aa4d1 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:46.794868 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42913b-3cf0-4592-a080-12cccc4aa4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:46.794868 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42913b-3cf0-4592-a080-12cccc4aa4d1 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:46.794868 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 32/62] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:46.800284 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abd2eb-0a35-45ca-839c-1e7894934666 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:46.800685 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abd2eb-0a35-45ca-839c-1e7894934666 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:46.800865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2abd2eb-0a35-45ca-839c-1e7894934666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:46.824937 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a2abd2eb-0a35-45ca-839c-1e7894934666 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:46.826144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a2abd2eb-0a35-45ca-839c-1e7894934666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:46.906593 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 31/63] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:54:46.941009 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b42913b-3cf0-4592-a080-12cccc4aa4d1 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:46.965614 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea00972-37ce-448c-bc5a-01a2ef7f126f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:46.965927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea00972-37ce-448c-bc5a-01a2ef7f126f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:46.966181 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea00972-37ce-448c-bc5a-01a2ef7f126f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:46.967683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 32/64] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:54:46.973161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcadf70-539a-4784-aa94-28b233de421d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:46.973452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcadf70-539a-4784-aa94-28b233de421d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:46.973975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcadf70-539a-4784-aa94-28b233de421d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:46.988245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:46.991986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-afcadf70-539a-4784-aa94-28b233de421d None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:46.993540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-afcadf70-539a-4784-aa94-28b233de421d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:47.063291 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 33/65] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 13:54:47.064563 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2355c-2aeb-4e0c-afc6-9d658d3aa19e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:47.065042 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2355c-2aeb-4e0c-afc6-9d658d3aa19e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:47.065486 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2355c-2aeb-4e0c-afc6-9d658d3aa19e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:47.067190 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 34/66] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:47.067875 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b42913b-3cf0-4592-a080-12cccc4aa4d1 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:47.068600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82a55b-408d-4efb-8895-c52e4b550100 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:47.069058 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82a55b-408d-4efb-8895-c52e4b550100 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:47.069489 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82a55b-408d-4efb-8895-c52e4b550100 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:47.070865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 35/67] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:47.077161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864c4d8-ae40-44fc-88cb-ff4e16fba999 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:47.077695 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864c4d8-ae40-44fc-88cb-ff4e16fba999 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:47.078522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864c4d8-ae40-44fc-88cb-ff4e16fba999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:47.096250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5864c4d8-ae40-44fc-88cb-ff4e16fba999 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:47.097482 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5864c4d8-ae40-44fc-88cb-ff4e16fba999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:47.099644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:47.108282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625564af-0022-40dd-bb6e-3c8ddee07acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:47.109381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625564af-0022-40dd-bb6e-3c8ddee07acb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:47.109889 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625564af-0022-40dd-bb6e-3c8ddee07acb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:47.110383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625564af-0022-40dd-bb6e-3c8ddee07acb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:47.121614 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625564af-0022-40dd-bb6e-3c8ddee07acb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:54:47.129241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-625564af-0022-40dd-bb6e-3c8ddee07acb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:54:47.129241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:54:47.129241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:54:47.129241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:54:47.129241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:54:47.129241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:54:47.129241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:54:47.129241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:54:47.129241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:54:47.129241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:54:47.129241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:54:47.129241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:54:47.129241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:54:47.129241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:54:47.129241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:54:47.129241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:54:47.130167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:54:47.130167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:54:47.130167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:54:47.130167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:54:47.130167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:54:47.130167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:54:47.130167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:54:47.130167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:54:47.130167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:54:47.130167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:54:47.130167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:54:47.130167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:54:47.130167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:54:47.130167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:54:47.130167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:54:47.130167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:54:47.130942 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:54:47.130942 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:54:47.130942 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:54:47.130942 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:54:47.130942 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:54:47.130942 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:54:47.130942 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:54:47.130942 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:54:47.130942 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:54:47.130942 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:54:47.130942 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:54:47.130942 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:54:47.130942 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 33/68] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:54:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:54:47.131619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171314e-ecd7-4980-baab-b62086e4075c None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:47.132077 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171314e-ecd7-4980-baab-b62086e4075c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:47.132487 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171314e-ecd7-4980-baab-b62086e4075c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:47.149580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-4171314e-ecd7-4980-baab-b62086e4075c None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:47.150953 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4171314e-ecd7-4980-baab-b62086e4075c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:47.175309 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 36/69] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 13:54:47.176812 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864c4d8-ae40-44fc-88cb-ff4e16fba999 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:47.221606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 34/70] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:54:47.222663 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171314e-ecd7-4980-baab-b62086e4075c None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:47.244598 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:47.296411 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864c4d8-ae40-44fc-88cb-ff4e16fba999 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:47.324396 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171314e-ecd7-4980-baab-b62086e4075c None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:47.333769 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ece567-e454-42ee-9042-79b3d95abc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:47.334878 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ece567-e454-42ee-9042-79b3d95abc0c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:47.335225 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ece567-e454-42ee-9042-79b3d95abc0c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:47.335626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ece567-e454-42ee-9042-79b3d95abc0c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:47.343741 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:47.349287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ece567-e454-42ee-9042-79b3d95abc0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:54:47.358448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-64ece567-e454-42ee-9042-79b3d95abc0c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:54:47.358448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:54:47.358448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:54:47.358448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:54:47.358448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:54:47.358448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:54:47.358448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:54:47.358448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:54:47.358448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:54:47.358448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:54:47.358448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:54:47.358448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:54:47.358448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:54:47.358448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:54:47.358448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:54:47.358448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:54:47.359295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:54:47.359295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:54:47.359295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:54:47.359295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:54:47.359295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:54:47.359295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:54:47.359295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:54:47.359295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:54:47.359295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:54:47.359295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:54:47.359295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:54:47.359295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:54:47.359295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:54:47.359295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:54:47.359295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:54:47.359295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:54:47.360501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:54:47.360501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:54:47.360501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:54:47.360501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:54:47.360501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:54:47.360501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:54:47.360501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:54:47.360501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:54:47.360501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:54:47.360501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:54:47.360501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:54:47.360501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:54:47.360501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 37/71] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:54:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:54:47.361424 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ece567-e454-42ee-9042-79b3d95abc0c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:47.365242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f077b-39fc-46a7-b1a5-ddb126719336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:47.365926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f077b-39fc-46a7-b1a5-ddb126719336 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:47.366219 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f077b-39fc-46a7-b1a5-ddb126719336 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:47.366500 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f077b-39fc-46a7-b1a5-ddb126719336 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:47.367253 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816f077b-39fc-46a7-b1a5-ddb126719336 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:47.367563 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816f077b-39fc-46a7-b1a5-ddb126719336 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:47.369023 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816f077b-39fc-46a7-b1a5-ddb126719336 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:47.382520 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 35/72] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:54:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:47.382520 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f077b-39fc-46a7-b1a5-ddb126719336 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:47.458875 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:47.470347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ece567-e454-42ee-9042-79b3d95abc0c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:47.478334 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f077b-39fc-46a7-b1a5-ddb126719336 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:47.503109 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03bc59-e651-4333-b995-b8e8510ef193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:47.503907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03bc59-e651-4333-b995-b8e8510ef193 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:47.504232 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03bc59-e651-4333-b995-b8e8510ef193 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:47.504517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03bc59-e651-4333-b995-b8e8510ef193 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:47.510414 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218359ef-741b-4ad4-9556-ebaf7cb89812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:47.511214 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218359ef-741b-4ad4-9556-ebaf7cb89812 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:47.511502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218359ef-741b-4ad4-9556-ebaf7cb89812 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:47.511792 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218359ef-741b-4ad4-9556-ebaf7cb89812 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:47.515559 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03bc59-e651-4333-b995-b8e8510ef193 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:54:47.521591 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218359ef-741b-4ad4-9556-ebaf7cb89812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:54:47.523277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-8b03bc59-e651-4333-b995-b8e8510ef193 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:54:47.523277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:54:47.523277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:54:47.523277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:54:47.523277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:54:47.523277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:54:47.523277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:54:47.523277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:54:47.523277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:54:47.523277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:54:47.523277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:54:47.523277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:54:47.523277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:54:47.523277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:54:47.523277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:54:47.523277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:54:47.524832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:54:47.524832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:54:47.524832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:54:47.524832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:54:47.524832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:54:47.524832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:54:47.524832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:54:47.524832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:54:47.524832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:54:47.524832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:54:47.524832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:54:47.524832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:54:47.524832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:54:47.524832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:54:47.524832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:54:47.524832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:54:47.525669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:54:47.525669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:54:47.525669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:54:47.525669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:54:47.525669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:54:47.525669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:54:47.525669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:54:47.525669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:54:47.525669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:54:47.525669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:54:47.525669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:54:47.525669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:54:47.525669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 38/73] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:54:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:54:47.526364 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03bc59-e651-4333-b995-b8e8510ef193 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:47.530017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-218359ef-741b-4ad4-9556-ebaf7cb89812 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:54:47.530017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:54:47.530017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:54:47.530017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:54:47.530017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:54:47.530017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:54:47.530017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:54:47.530017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:54:47.530017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:54:47.530017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:54:47.530017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:54:47.530017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:54:47.530017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:54:47.530017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:54:47.530017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:54:47.530017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:54:47.530937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:54:47.530937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:54:47.530937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:54:47.530937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:54:47.530937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:54:47.530937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:54:47.530937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:54:47.530937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:54:47.530937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:54:47.530937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:54:47.530937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:54:47.530937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:54:47.530937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:54:47.530937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:54:47.530937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:54:47.530937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:54:47.531674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:54:47.531674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:54:47.531674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:54:47.531674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:54:47.531674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:54:47.531674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:54:47.531674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:54:47.531674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:54:47.531674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:54:47.531674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:54:47.531674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:54:47.531674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:54:47.531674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 36/74] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Jul 18 13:54:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:54:47.532531 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218359ef-741b-4ad4-9556-ebaf7cb89812 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:47.589061 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:47.634711 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218359ef-741b-4ad4-9556-ebaf7cb89812 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:47.638272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03bc59-e651-4333-b995-b8e8510ef193 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:47.668594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81adb2-1936-4b56-be1a-b7869b3009e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:47.669476 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81adb2-1936-4b56-be1a-b7869b3009e6 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:47.669775 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81adb2-1936-4b56-be1a-b7869b3009e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:47.670063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81adb2-1936-4b56-be1a-b7869b3009e6 None admin] PATH_INFO: `/v3/users/7a10bd46534e415080cc3fb7cb29bb4d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:47.671254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81adb2-1936-4b56-be1a-b7869b3009e6 None admin] RBAC: Authorizing `identity:get_user(user_id=7a10bd46534e415080cc3fb7cb29bb4d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:47.672748 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81adb2-1936-4b56-be1a-b7869b3009e6 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:47.673822 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c42e8b-145b-4224-8d66-f651681fe26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:47.674387 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 37/75] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:54:47 2022] GET /identity/v3/users/7a10bd46534e415080cc3fb7cb29bb4d => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:47.675572 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28811183-5570-42e9-b968-69af51e75dcc None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:47.675760 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c42e8b-145b-4224-8d66-f651681fe26d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:47.675867 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28811183-5570-42e9-b968-69af51e75dcc None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:47.676128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c42e8b-145b-4224-8d66-f651681fe26d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:47.676222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28811183-5570-42e9-b968-69af51e75dcc None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:47.676426 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c42e8b-145b-4224-8d66-f651681fe26d None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:47.677201 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c42e8b-145b-4224-8d66-f651681fe26d None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:47.677381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 38/76] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:47.677525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c42e8b-145b-4224-8d66-f651681fe26d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:47.678115 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28811183-5570-42e9-b968-69af51e75dcc None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:47.679241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c42e8b-145b-4224-8d66-f651681fe26d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:47.693220 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 39/77] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:54:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:47.693220 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c42e8b-145b-4224-8d66-f651681fe26d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:47.695487 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:47.783926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28811183-5570-42e9-b968-69af51e75dcc None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:47.787725 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c42e8b-145b-4224-8d66-f651681fe26d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:47.808298 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:47.822699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d233c1-d46f-428a-87a1-bc5c317cf5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:47.823374 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d233c1-d46f-428a-87a1-bc5c317cf5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:47.823669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d233c1-d46f-428a-87a1-bc5c317cf5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:47.823968 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d233c1-d46f-428a-87a1-bc5c317cf5c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:47.824731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d233c1-d46f-428a-87a1-bc5c317cf5c4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:47.825048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d233c1-d46f-428a-87a1-bc5c317cf5c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:47.825285 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0f437-6e83-4809-9746-45543698d977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:47.825955 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0f437-6e83-4809-9746-45543698d977 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:47.826243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0f437-6e83-4809-9746-45543698d977 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:47.826463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d233c1-d46f-428a-87a1-bc5c317cf5c4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:47.826549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0f437-6e83-4809-9746-45543698d977 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:47.827248 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0f437-6e83-4809-9746-45543698d977 None admin] RBAC: Adding query filter params (name=member) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:47.827557 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0f437-6e83-4809-9746-45543698d977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:47.829304 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0f437-6e83-4809-9746-45543698d977 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:47.834747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 40/78] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:54:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:47.839552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d233c1-d46f-428a-87a1-bc5c317cf5c4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:47.839675 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 39/79] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:54:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:47.839675 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19b771-4197-4cf9-9f8a-cd9ba44842be None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:47.839675 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19b771-4197-4cf9-9f8a-cd9ba44842be None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:47.839900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19b771-4197-4cf9-9f8a-cd9ba44842be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:47.855522 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ca19b771-4197-4cf9-9f8a-cd9ba44842be None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:47.856755 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ca19b771-4197-4cf9-9f8a-cd9ba44842be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:47.921932 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 40/80] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:54:47.922756 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca19b771-4197-4cf9-9f8a-cd9ba44842be None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:47.924887 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:47.950144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d233c1-d46f-428a-87a1-bc5c317cf5c4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:47.982991 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a31424-ae17-4a85-86c4-d1bcb55ba5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:47.983652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a31424-ae17-4a85-86c4-d1bcb55ba5e7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:47.983961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a31424-ae17-4a85-86c4-d1bcb55ba5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:47.984247 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a31424-ae17-4a85-86c4-d1bcb55ba5e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:48.028376 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca19b771-4197-4cf9-9f8a-cd9ba44842be None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:48.032522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:48.051136 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a31424-ae17-4a85-86c4-d1bcb55ba5e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:48.052688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a31424-ae17-4a85-86c4-d1bcb55ba5e7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:48.062177 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c2909-a75c-4cb5-9285-694687d515f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:48.063058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c2909-a75c-4cb5-9285-694687d515f9 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:48.063371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c2909-a75c-4cb5-9285-694687d515f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:48.063653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c2909-a75c-4cb5-9285-694687d515f9 None admin] PATH_INFO: `/v3/users/2ecf077c36454bf782f946577207b833` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:48.076635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c2909-a75c-4cb5-9285-694687d515f9 None admin] RBAC: Authorizing `identity:get_user(user_id=2ecf077c36454bf782f946577207b833)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:48.078352 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c2909-a75c-4cb5-9285-694687d515f9 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:48.080054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 41/81] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:54:47 2022] GET /identity/v3/users/2ecf077c36454bf782f946577207b833 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:48.080825 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c2909-a75c-4cb5-9285-694687d515f9 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:48.139491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:48.150632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 41/82] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:54:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:54:48.151476 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a31424-ae17-4a85-86c4-d1bcb55ba5e7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:48.182153 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c2909-a75c-4cb5-9285-694687d515f9 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:48.216548 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b8082-ba89-40e0-b80f-e8737ec7b5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:48.217421 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b8082-ba89-40e0-b80f-e8737ec7b5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:48.217719 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b8082-ba89-40e0-b80f-e8737ec7b5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:48.218002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b8082-ba89-40e0-b80f-e8737ec7b5d1 None admin] PATH_INFO: `/v3/users/2ecf077c36454bf782f946577207b833` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:48.219215 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b8082-ba89-40e0-b80f-e8737ec7b5d1 None admin] RBAC: Authorizing `identity:get_user(user_id=2ecf077c36454bf782f946577207b833)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:48.220715 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501b8082-ba89-40e0-b80f-e8737ec7b5d1 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:48.222353 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 42/83] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:54:48 2022] GET /identity/v3/users/2ecf077c36454bf782f946577207b833 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:48.223115 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b8082-ba89-40e0-b80f-e8737ec7b5d1 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:48.231006 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:48.249334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a31424-ae17-4a85-86c4-d1bcb55ba5e7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:48.282489 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21246892-d7ff-4765-a7e2-f455dce6490a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:48.283356 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21246892-d7ff-4765-a7e2-f455dce6490a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:48.283650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21246892-d7ff-4765-a7e2-f455dce6490a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:48.283931 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21246892-d7ff-4765-a7e2-f455dce6490a None admin] PATH_INFO: `/v3/users/2ecf077c36454bf782f946577207b833` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:48.285388 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21246892-d7ff-4765-a7e2-f455dce6490a None admin] RBAC: Authorizing `identity:get_user(user_id=2ecf077c36454bf782f946577207b833)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:48.286843 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21246892-d7ff-4765-a7e2-f455dce6490a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:48.288501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 42/84] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:54:48 2022] GET /identity/v3/users/2ecf077c36454bf782f946577207b833 => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:48.289261 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21246892-d7ff-4765-a7e2-f455dce6490a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:48.325309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b8082-ba89-40e0-b80f-e8737ec7b5d1 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:48.351390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:48.359966 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98a3f6-42da-4044-993f-157e1e742ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:48.360778 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98a3f6-42da-4044-993f-157e1e742ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:48.361069 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98a3f6-42da-4044-993f-157e1e742ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:48.361345 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98a3f6-42da-4044-993f-157e1e742ee0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:48.362514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98a3f6-42da-4044-993f-157e1e742ee0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:48.363990 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98a3f6-42da-4044-993f-157e1e742ee0 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:48.365594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 43/85] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:54:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:48.366350 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98a3f6-42da-4044-993f-157e1e742ee0 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:48.389916 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21246892-d7ff-4765-a7e2-f455dce6490a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:48.423431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec705e-eff6-40c0-b4de-01e3a7fb63b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:48.424115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec705e-eff6-40c0-b4de-01e3a7fb63b5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:48.424415 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec705e-eff6-40c0-b4de-01e3a7fb63b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:48.424703 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec705e-eff6-40c0-b4de-01e3a7fb63b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:48.443550 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:48.466017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98a3f6-42da-4044-993f-157e1e742ee0 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:48.492926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec705e-eff6-40c0-b4de-01e3a7fb63b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:48.495005 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec705e-eff6-40c0-b4de-01e3a7fb63b5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:48.499665 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcec59d-fde0-4a89-99e5-083d2d731b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:48.500555 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcec59d-fde0-4a89-99e5-083d2d731b3d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:48.500852 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcec59d-fde0-4a89-99e5-083d2d731b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:48.501139 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcec59d-fde0-4a89-99e5-083d2d731b3d None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:48.502348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcec59d-fde0-4a89-99e5-083d2d731b3d None admin] RBAC: Authorizing `identity:get_project(project_id=ec42cd390d9e4d07bb7573020b5b1003)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:48.503788 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcec59d-fde0-4a89-99e5-083d2d731b3d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:48.505687 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 44/86] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:54:48 2022] GET /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:48.506379 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcec59d-fde0-4a89-99e5-083d2d731b3d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:48.566926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:48.596335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 43/87] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:54:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:54:48.597246 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec705e-eff6-40c0-b4de-01e3a7fb63b5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:48.605960 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcec59d-fde0-4a89-99e5-083d2d731b3d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:48.639236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493f0ae-55a1-4a43-9f15-309e9e49bc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:48.639893 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493f0ae-55a1-4a43-9f15-309e9e49bc2f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:48.640279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493f0ae-55a1-4a43-9f15-309e9e49bc2f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:48.640515 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493f0ae-55a1-4a43-9f15-309e9e49bc2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:48.648110 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:48.698653 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec705e-eff6-40c0-b4de-01e3a7fb63b5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:48.708958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493f0ae-55a1-4a43-9f15-309e9e49bc2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:48.710565 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493f0ae-55a1-4a43-9f15-309e9e49bc2f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:48.731414 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ad8d9-b4f6-48e9-991a-724a4244a66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:48.732099 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ad8d9-b4f6-48e9-991a-724a4244a66b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:48.732395 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ad8d9-b4f6-48e9-991a-724a4244a66b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:48.732693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ad8d9-b4f6-48e9-991a-724a4244a66b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:48.768934 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:48.798689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ad8d9-b4f6-48e9-991a-724a4244a66b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:48.800249 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ad8d9-b4f6-48e9-991a-724a4244a66b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:48.807896 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 45/88] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:54:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:54:48.808730 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493f0ae-55a1-4a43-9f15-309e9e49bc2f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:48.850690 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:48.901649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 44/89] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:54:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:54:48.902406 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ad8d9-b4f6-48e9-991a-724a4244a66b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:48.910214 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493f0ae-55a1-4a43-9f15-309e9e49bc2f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:48.946560 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5249389-5fd3-4ea6-836c-6e1a853acb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:48.947244 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5249389-5fd3-4ea6-836c-6e1a853acb2c None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:48.947905 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5249389-5fd3-4ea6-836c-6e1a853acb2c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:48.947905 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5249389-5fd3-4ea6-836c-6e1a853acb2c None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:48.948539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5249389-5fd3-4ea6-836c-6e1a853acb2c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:48.949995 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5249389-5fd3-4ea6-836c-6e1a853acb2c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:48.975707 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 46/90] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Jul 18 13:54:48 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:54:48.980926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5249389-5fd3-4ea6-836c-6e1a853acb2c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:49.006454 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ad8d9-b4f6-48e9-991a-724a4244a66b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:49.006996 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:49.049656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5f42a-f291-4998-a585-91db0e65cbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:49.050351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5f42a-f291-4998-a585-91db0e65cbe3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:49.050639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5f42a-f291-4998-a585-91db0e65cbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:49.050924 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5f42a-f291-4998-a585-91db0e65cbe3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:49.051787 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd5f42a-f291-4998-a585-91db0e65cbe3 None admin] RBAC: Adding query filter params (role.id=c60a99de086f45c6820f7872a6b43feb, scope.project.id=ec42cd390d9e4d07bb7573020b5b1003, user.id=7a10bd46534e415080cc3fb7cb29bb4d) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:49.052142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd5f42a-f291-4998-a585-91db0e65cbe3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:49.053642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd5f42a-f291-4998-a585-91db0e65cbe3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:49.063701 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 45/91] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:54:48 2022] GET /identity/v3/role_assignments?user.id=7a10bd46534e415080cc3fb7cb29bb4d&scope.project.id=ec42cd390d9e4d07bb7573020b5b1003&role.id=c60a99de086f45c6820f7872a6b43feb => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:49.069949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5f42a-f291-4998-a585-91db0e65cbe3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:49.099000 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:49.119732 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5249389-5fd3-4ea6-836c-6e1a853acb2c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:49.168995 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc074e-e285-4bb4-981b-ff022c825b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:49.169797 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc074e-e285-4bb4-981b-ff022c825b2b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:49.170084 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc074e-e285-4bb4-981b-ff022c825b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:49.170362 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc074e-e285-4bb4-981b-ff022c825b2b None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:49.171601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bc074e-e285-4bb4-981b-ff022c825b2b None admin] RBAC: Authorizing `identity:get_project(project_id=ec42cd390d9e4d07bb7573020b5b1003)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:49.174910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bc074e-e285-4bb4-981b-ff022c825b2b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:49.174910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 47/92] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:54:48 2022] GET /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:49.175721 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc074e-e285-4bb4-981b-ff022c825b2b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:49.222054 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5f42a-f291-4998-a585-91db0e65cbe3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:49.268999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a76f9-ac5d-4822-be9b-3ca191635e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:49.269879 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a76f9-ac5d-4822-be9b-3ca191635e6d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:49.270174 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a76f9-ac5d-4822-be9b-3ca191635e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:49.270456 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a76f9-ac5d-4822-be9b-3ca191635e6d None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:49.275129 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18a76f9-ac5d-4822-be9b-3ca191635e6d None admin] RBAC: Authorizing `identity:get_project(project_id=ec42cd390d9e4d07bb7573020b5b1003)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:49.275129 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18a76f9-ac5d-4822-be9b-3ca191635e6d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:49.275843 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 46/93] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:54:49 2022] GET /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:49.280878 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a76f9-ac5d-4822-be9b-3ca191635e6d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:49.299846 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:49.306672 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc074e-e285-4bb4-981b-ff022c825b2b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:49.351732 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd7fd9-5572-4db9-abd8-a6eb23b9c322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:49.353303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd7fd9-5572-4db9-abd8-a6eb23b9c322 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:49.353700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd7fd9-5572-4db9-abd8-a6eb23b9c322 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:49.354106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd7fd9-5572-4db9-abd8-a6eb23b9c322 None admin] PATH_INFO: `/v3/projects/d045f37f21894bc083e89f3fed0768d5` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:49.355392 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd7fd9-5572-4db9-abd8-a6eb23b9c322 None admin] RBAC: Authorizing `identity:get_project(project_id=d045f37f21894bc083e89f3fed0768d5)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:49.356980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd7fd9-5572-4db9-abd8-a6eb23b9c322 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:49.358956 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 48/94] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:54:49 2022] GET /identity/v3/projects/d045f37f21894bc083e89f3fed0768d5 => generated 312 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:49.359692 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd7fd9-5572-4db9-abd8-a6eb23b9c322 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:49.386083 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:49.407654 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a76f9-ac5d-4822-be9b-3ca191635e6d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:49.445767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8825ffa-e80b-45ce-9e15-684e6ea684a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:49.446483 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8825ffa-e80b-45ce-9e15-684e6ea684a8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:49.446780 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8825ffa-e80b-45ce-9e15-684e6ea684a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:49.447069 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8825ffa-e80b-45ce-9e15-684e6ea684a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:49.447921 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8825ffa-e80b-45ce-9e15-684e6ea684a8 None admin] RBAC: Adding query filter params (role.id=1399a838106b4f9bb5fa1c942cc3ef22, scope.project.id=ec42cd390d9e4d07bb7573020b5b1003, user.id=2ecf077c36454bf782f946577207b833) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:49.448273 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8825ffa-e80b-45ce-9e15-684e6ea684a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:49.449874 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8825ffa-e80b-45ce-9e15-684e6ea684a8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:49.456891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 47/95] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:54:49 2022] GET /identity/v3/role_assignments?user.id=2ecf077c36454bf782f946577207b833&scope.project.id=ec42cd390d9e4d07bb7573020b5b1003&role.id=1399a838106b4f9bb5fa1c942cc3ef22 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:49.457817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8825ffa-e80b-45ce-9e15-684e6ea684a8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:49.497229 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd7fd9-5572-4db9-abd8-a6eb23b9c322 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:49.566759 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59ed79-810c-4f42-a98f-9f885c9a8969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:49.567450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59ed79-810c-4f42-a98f-9f885c9a8969 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:49.567743 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59ed79-810c-4f42-a98f-9f885c9a8969 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:49.572169 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59ed79-810c-4f42-a98f-9f885c9a8969 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:49.573202 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59ed79-810c-4f42-a98f-9f885c9a8969 None admin] RBAC: Adding query filter params (role.id=580aae575f414b3c9b4ccf1e27e0b640, scope.project.id=ec42cd390d9e4d07bb7573020b5b1003, user.id=2ecf077c36454bf782f946577207b833) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:49.573538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59ed79-810c-4f42-a98f-9f885c9a8969 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:49.575067 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59ed79-810c-4f42-a98f-9f885c9a8969 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:49.593383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 49/96] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:54:49 2022] GET /identity/v3/role_assignments?user.id=2ecf077c36454bf782f946577207b833&scope.project.id=ec42cd390d9e4d07bb7573020b5b1003&role.id=580aae575f414b3c9b4ccf1e27e0b640 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:49.678319 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8825ffa-e80b-45ce-9e15-684e6ea684a8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:49.732817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5129f2a-cf6d-4a91-87f5-fac67ee24769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:49.733434 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5129f2a-cf6d-4a91-87f5-fac67ee24769 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:49.733729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5129f2a-cf6d-4a91-87f5-fac67ee24769 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:49.734013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5129f2a-cf6d-4a91-87f5-fac67ee24769 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:49.734888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5129f2a-cf6d-4a91-87f5-fac67ee24769 None admin] RBAC: Adding query filter params (role.id=1399a838106b4f9bb5fa1c942cc3ef22, scope.project.id=d045f37f21894bc083e89f3fed0768d5, user.id=2ecf077c36454bf782f946577207b833) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:49.735244 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5129f2a-cf6d-4a91-87f5-fac67ee24769 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:49.736897 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5129f2a-cf6d-4a91-87f5-fac67ee24769 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:49.744002 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 48/97] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:54:49 2022] GET /identity/v3/role_assignments?user.id=2ecf077c36454bf782f946577207b833&scope.project.id=d045f37f21894bc083e89f3fed0768d5&role.id=1399a838106b4f9bb5fa1c942cc3ef22 => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:51.515304 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d87d60-b647-4e65-b952-f9950a957336 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:51.515808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d87d60-b647-4e65-b952-f9950a957336 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:51.515881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d87d60-b647-4e65-b952-f9950a957336 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:51.517424 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 50/98] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:51.524960 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c495d55-a466-4c0d-ad3c-11f79449601e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:51.524960 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c495d55-a466-4c0d-ad3c-11f79449601e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:51.524960 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c495d55-a466-4c0d-ad3c-11f79449601e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:51.548993 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8c495d55-a466-4c0d-ad3c-11f79449601e None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:51.550534 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8c495d55-a466-4c0d-ad3c-11f79449601e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:51.554712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:51.645496 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 49/99] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:54:51.681332 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d87d60-b647-4e65-b952-f9950a957336 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:51.715233 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:51.807141 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d87d60-b647-4e65-b952-f9950a957336 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:51.842495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f03dba-e7d1-4b30-884f-0b4650bc1ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:51.843630 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f03dba-e7d1-4b30-884f-0b4650bc1ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:51.844449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f03dba-e7d1-4b30-884f-0b4650bc1ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:51.844987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f03dba-e7d1-4b30-884f-0b4650bc1ef4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:51.846374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f03dba-e7d1-4b30-884f-0b4650bc1ef4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:51.848059 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f03dba-e7d1-4b30-884f-0b4650bc1ef4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:51.849875 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 51/100] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:54:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:51.861065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c495d55-a466-4c0d-ad3c-11f79449601e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:52.036615 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c495d55-a466-4c0d-ad3c-11f79449601e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:52.064513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915aec7-f37a-4c01-b85e-7d99380098c3 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:52.064786 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915aec7-f37a-4c01-b85e-7d99380098c3 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:52.065007 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915aec7-f37a-4c01-b85e-7d99380098c3 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:52.066391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 52/101] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:52.071812 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87a73f-3032-439c-ac34-568a373359cb None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:52.072153 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87a73f-3032-439c-ac34-568a373359cb None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:52.072440 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87a73f-3032-439c-ac34-568a373359cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:52.097857 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb4b25-8133-4ae2-9fee-111dd8e3b8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:52.098538 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb4b25-8133-4ae2-9fee-111dd8e3b8e0 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:52.098830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb4b25-8133-4ae2-9fee-111dd8e3b8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:52.099115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb4b25-8133-4ae2-9fee-111dd8e3b8e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:52.099842 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb4b25-8133-4ae2-9fee-111dd8e3b8e0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:52.101349 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb4b25-8133-4ae2-9fee-111dd8e3b8e0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:52.103848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-3c87a73f-3032-439c-ac34-568a373359cb None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:52.105089 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3c87a73f-3032-439c-ac34-568a373359cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:52.122929 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:52.155176 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 50/102] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Jul 18 13:54:51 2022] POST /identity/v3/projects => generated 302 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:54:52.157323 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32ed9a-4291-4721-a7a1-4886684a7293 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:52.158142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32ed9a-4291-4721-a7a1-4886684a7293 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:52.158618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32ed9a-4291-4721-a7a1-4886684a7293 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:52.163713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 51/103] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:52.177319 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8eeb5f-119c-45fe-808a-d4778db07174 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:52.177931 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8eeb5f-119c-45fe-808a-d4778db07174 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:52.178408 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8eeb5f-119c-45fe-808a-d4778db07174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:52.217020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6f8eeb5f-119c-45fe-808a-d4778db07174 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:52.218708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6f8eeb5f-119c-45fe-808a-d4778db07174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:52.263645 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 53/104] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 13:54:52.301067 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87a73f-3032-439c-ac34-568a373359cb None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:52.334682 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 52/105] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 18 13:54:52.341888 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:52.382048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8eeb5f-119c-45fe-808a-d4778db07174 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:52.490713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c87a73f-3032-439c-ac34-568a373359cb None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:52.516562 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8eeb5f-119c-45fe-808a-d4778db07174 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:52.534678 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd27cc-22e4-4612-839b-7523bb984c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:52.535499 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd27cc-22e4-4612-839b-7523bb984c83 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:52.535794 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd27cc-22e4-4612-839b-7523bb984c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:52.536115 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd27cc-22e4-4612-839b-7523bb984c83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:52.545273 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:52.553956 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd27cc-22e4-4612-839b-7523bb984c83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:54:52.561838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee432fb-819f-4d2b-b1ac-0f4d62a0ff1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:52.562875 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee432fb-819f-4d2b-b1ac-0f4d62a0ff1d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:52.563351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee432fb-819f-4d2b-b1ac-0f4d62a0ff1d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:52.563698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-73dd27cc-22e4-4612-839b-7523bb984c83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:54:52.563698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:54:52.563698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:54:52.563698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:54:52.563698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:54:52.563698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:54:52.563698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:54:52.563698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:54:52.563698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:54:52.563698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:54:52.563698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:54:52.563698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:54:52.563698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:54:52.563698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:54:52.563698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:54:52.563698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:54:52.564553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:54:52.564553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:54:52.564553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:54:52.564553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:54:52.564553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:54:52.564553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:54:52.564553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:54:52.564553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:54:52.564553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:54:52.564553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:54:52.564553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:54:52.564553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:54:52.564553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:54:52.564553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:54:52.564553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:54:52.564553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:54:52.565309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:54:52.565309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:54:52.565309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:54:52.565309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:54:52.565309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:54:52.565309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:54:52.565309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:54:52.565309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:54:52.565309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:54:52.565309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:54:52.565309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:54:52.565309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:54:52.565309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 54/106] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:54:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:54:52.566669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6706146-8e6a-4a24-9ff4-4736e53dff7a None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:52.566831 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6706146-8e6a-4a24-9ff4-4736e53dff7a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:52.567121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6706146-8e6a-4a24-9ff4-4736e53dff7a None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:52.567329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee432fb-819f-4d2b-b1ac-0f4d62a0ff1d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:52.568600 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 55/107] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:52.573288 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6706146-8e6a-4a24-9ff4-4736e53dff7a None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:52.581708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee432fb-819f-4d2b-b1ac-0f4d62a0ff1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:54:52.590907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-eee432fb-819f-4d2b-b1ac-0f4d62a0ff1d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 13:54:52.590907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:54:52.590907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:54:52.590907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:54:52.590907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:54:52.590907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:54:52.590907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:54:52.590907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:54:52.590907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:54:52.590907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:54:52.590907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:54:52.590907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:54:52.590907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:54:52.590907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:54:52.590907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:54:52.590907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:54:52.591762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:54:52.591762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:54:52.591762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:54:52.591762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:54:52.591762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:54:52.591762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:54:52.591762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:54:52.591762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:54:52.591762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:54:52.591762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:54:52.591762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:54:52.591762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:54:52.591762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:54:52.591762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:54:52.591762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:54:52.591762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:54:52.592846 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:54:52.592846 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:54:52.592846 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:54:52.592846 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:54:52.592846 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:54:52.592846 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:54:52.592846 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:54:52.592846 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:54:52.592846 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:54:52.592846 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:54:52.592846 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 13:54:52.592846 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:54:52.595004 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 53/108] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:54:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:54:52.596506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99449d-9b60-477d-b786-2f2a0bb02b54 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:52.596972 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99449d-9b60-477d-b786-2f2a0bb02b54 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:52.597422 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99449d-9b60-477d-b786-2f2a0bb02b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:52.620487 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7a99449d-9b60-477d-b786-2f2a0bb02b54 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:52.621806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7a99449d-9b60-477d-b786-2f2a0bb02b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:52.666277 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:52.692174 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6706146-8e6a-4a24-9ff4-4736e53dff7a None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:52.698715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 54/109] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:54:52.703657 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99449d-9b60-477d-b786-2f2a0bb02b54 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:52.740634 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bccaad-a56b-4899-b992-e7746d8b9d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:52.741646 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bccaad-a56b-4899-b992-e7746d8b9d29 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:52.742293 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bccaad-a56b-4899-b992-e7746d8b9d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:52.743904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bccaad-a56b-4899-b992-e7746d8b9d29 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:52.744856 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bccaad-a56b-4899-b992-e7746d8b9d29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:52.745534 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bccaad-a56b-4899-b992-e7746d8b9d29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:52.747820 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bccaad-a56b-4899-b992-e7746d8b9d29 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:52.761851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 56/110] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:54:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:52.765132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bccaad-a56b-4899-b992-e7746d8b9d29 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:52.879828 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99449d-9b60-477d-b786-2f2a0bb02b54 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:52.907129 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:52.933675 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e8a76-336b-4c90-a4c1-f8b44d34b0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:52.934677 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e8a76-336b-4c90-a4c1-f8b44d34b0ea None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:52.935138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e8a76-336b-4c90-a4c1-f8b44d34b0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:52.935238 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bccaad-a56b-4899-b992-e7746d8b9d29 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:52.935676 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e8a76-336b-4c90-a4c1-f8b44d34b0ea None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:52.936948 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e8a76-336b-4c90-a4c1-f8b44d34b0ea None admin] RBAC: Adding query filter params (name=reader) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:52.938261 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e8a76-336b-4c90-a4c1-f8b44d34b0ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:52.939873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e8a76-336b-4c90-a4c1-f8b44d34b0ea None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:52.950548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 55/111] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:54:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:52.956235 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e8a76-336b-4c90-a4c1-f8b44d34b0ea None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:52.981260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1327c9-2072-419f-98c7-4a52a4322a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:52.982325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1327c9-2072-419f-98c7-4a52a4322a8f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:52.982759 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1327c9-2072-419f-98c7-4a52a4322a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:52.983183 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1327c9-2072-419f-98c7-4a52a4322a8f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:52.994152 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:53.000347 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1327c9-2072-419f-98c7-4a52a4322a8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:54:53.010828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-7a1327c9-2072-419f-98c7-4a52a4322a8f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:54:53.010828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:54:53.010828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:54:53.010828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:54:53.010828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:54:53.010828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:54:53.010828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:54:53.010828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:54:53.010828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:54:53.010828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:54:53.010828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:54:53.010828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:54:53.010828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:54:53.010828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:54:53.010828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:54:53.010828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:54:53.011903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:54:53.011903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:54:53.011903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:54:53.011903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:54:53.011903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:54:53.011903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:54:53.011903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:54:53.011903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:54:53.011903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:54:53.011903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:54:53.011903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:54:53.011903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:54:53.011903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:54:53.011903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:54:53.011903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:54:53.011903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:54:53.012900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:54:53.012900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:54:53.012900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:54:53.012900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:54:53.012900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:54:53.012900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:54:53.012900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:54:53.012900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:54:53.012900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:54:53.012900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:54:53.012900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:54:53.012900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:54:53.015845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 57/112] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:54:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:54:53.017358 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8ed32-604a-4d4d-bd3a-ec1ef41b638a None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:53.017797 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8ed32-604a-4d4d-bd3a-ec1ef41b638a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:53.018204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8ed32-604a-4d4d-bd3a-ec1ef41b638a None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:53.019849 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 58/113] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:53.021026 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8ed32-604a-4d4d-bd3a-ec1ef41b638a None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:53.079828 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e8a76-336b-4c90-a4c1-f8b44d34b0ea None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:53.118137 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9943aaa-b93b-4683-8f9a-9d16697bd91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:53.119446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9943aaa-b93b-4683-8f9a-9d16697bd91a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:53.119804 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9943aaa-b93b-4683-8f9a-9d16697bd91a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:53.120246 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9943aaa-b93b-4683-8f9a-9d16697bd91a None admin] PATH_INFO: `/v3/users/7a10bd46534e415080cc3fb7cb29bb4d` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:53.121488 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9943aaa-b93b-4683-8f9a-9d16697bd91a None admin] RBAC: Authorizing `identity:get_user(user_id=7a10bd46534e415080cc3fb7cb29bb4d)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:53.123035 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9943aaa-b93b-4683-8f9a-9d16697bd91a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:53.128932 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 56/114] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:54:52 2022] GET /identity/v3/users/7a10bd46534e415080cc3fb7cb29bb4d => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:53.128932 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146931a-8dce-4a42-8dbd-d55ba56d2fe7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:53.128932 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146931a-8dce-4a42-8dbd-d55ba56d2fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:53.128932 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146931a-8dce-4a42-8dbd-d55ba56d2fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:53.143723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f146931a-8dce-4a42-8dbd-d55ba56d2fe7 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:53.144874 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f146931a-8dce-4a42-8dbd-d55ba56d2fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:53.146791 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8ed32-604a-4d4d-bd3a-ec1ef41b638a None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:53.182728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:53.183083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b652f7-3db5-4ceb-832d-1ce89a4db80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:53.183985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b652f7-3db5-4ceb-832d-1ce89a4db80f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:53.184288 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b652f7-3db5-4ceb-832d-1ce89a4db80f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:53.184639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b652f7-3db5-4ceb-832d-1ce89a4db80f None admin] PATH_INFO: `/v3/users/f2f07f4728f74a2fb27ea83e4cc23351` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:53.199837 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b652f7-3db5-4ceb-832d-1ce89a4db80f None admin] RBAC: Authorizing `identity:get_user(user_id=f2f07f4728f74a2fb27ea83e4cc23351)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:53.201333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b652f7-3db5-4ceb-832d-1ce89a4db80f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:53.203437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 59/115] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:54:53 2022] GET /identity/v3/users/f2f07f4728f74a2fb27ea83e4cc23351 => generated 289 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:53.204040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b652f7-3db5-4ceb-832d-1ce89a4db80f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:53.220242 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 57/116] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:54:53.221653 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d19da-d3c8-4621-8a89-5dfb2c934875 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:53.221955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d19da-d3c8-4621-8a89-5dfb2c934875 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:53.222242 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d19da-d3c8-4621-8a89-5dfb2c934875 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:53.223712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 58/117] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:53.232042 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d19da-d3c8-4621-8a89-5dfb2c934875 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:53.267659 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:53.311850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b652f7-3db5-4ceb-832d-1ce89a4db80f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:53.346774 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d19da-d3c8-4621-8a89-5dfb2c934875 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:53.350646 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62d15b-eebf-4f74-ab83-8f290e537990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:53.351322 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62d15b-eebf-4f74-ab83-8f290e537990 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:53.351611 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62d15b-eebf-4f74-ab83-8f290e537990 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:53.351893 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62d15b-eebf-4f74-ab83-8f290e537990 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:53.353490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee62d15b-eebf-4f74-ab83-8f290e537990 None admin] RBAC: Adding query filter params (name=member) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:53.353801 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee62d15b-eebf-4f74-ab83-8f290e537990 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:53.355551 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee62d15b-eebf-4f74-ab83-8f290e537990 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:53.364309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 60/118] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:54:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:53.365030 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62d15b-eebf-4f74-ab83-8f290e537990 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:53.384995 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c328dd-4d79-47ab-bc5e-50023ac08500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:53.386340 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c328dd-4d79-47ab-bc5e-50023ac08500 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:53.386817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c328dd-4d79-47ab-bc5e-50023ac08500 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:53.387293 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c328dd-4d79-47ab-bc5e-50023ac08500 None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:53.389261 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c328dd-4d79-47ab-bc5e-50023ac08500 None admin] RBAC: Authorizing `identity:get_project(project_id=ec42cd390d9e4d07bb7573020b5b1003)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:53.391612 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c328dd-4d79-47ab-bc5e-50023ac08500 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:53.394958 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 59/119] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:54:53 2022] GET /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:53.397095 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd31d32-1a62-41ee-906d-0cd8b7112f16 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:53.397631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd31d32-1a62-41ee-906d-0cd8b7112f16 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:53.398167 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd31d32-1a62-41ee-906d-0cd8b7112f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:53.417427 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6dd31d32-1a62-41ee-906d-0cd8b7112f16 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:53.418640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6dd31d32-1a62-41ee-906d-0cd8b7112f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:53.455770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:53.476491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62d15b-eebf-4f74-ab83-8f290e537990 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:53.490275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 60/120] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:54:53.497642 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:53.500800 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd31d32-1a62-41ee-906d-0cd8b7112f16 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:53.513055 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b9ac0-4780-44db-9ecb-3edb97da5901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:53.513906 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b9ac0-4780-44db-9ecb-3edb97da5901 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:53.514336 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b9ac0-4780-44db-9ecb-3edb97da5901 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:53.514753 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b9ac0-4780-44db-9ecb-3edb97da5901 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:53.588092 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b9ac0-4780-44db-9ecb-3edb97da5901 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:53.590535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b9ac0-4780-44db-9ecb-3edb97da5901 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:53.611313 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd31d32-1a62-41ee-906d-0cd8b7112f16 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:53.647817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45afb35d-69e0-4966-bafe-337298374e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:53.649010 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afb35d-69e0-4966-bafe-337298374e52 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:53.649344 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afb35d-69e0-4966-bafe-337298374e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:53.649635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afb35d-69e0-4966-bafe-337298374e52 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:53.660288 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45afb35d-69e0-4966-bafe-337298374e52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:54:53.669511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-45afb35d-69e0-4966-bafe-337298374e52 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:54:53.669511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:54:53.669511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:54:53.669511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:54:53.669511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:54:53.669511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:54:53.669511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:54:53.669511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:54:53.669511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:54:53.669511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:54:53.669511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:54:53.669511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:54:53.669511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:54:53.669511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:54:53.669511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:54:53.669511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:54:53.670379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:54:53.670379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:54:53.670379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:54:53.670379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:54:53.670379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:54:53.670379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:54:53.670379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:54:53.670379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:54:53.670379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:54:53.670379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:54:53.670379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:54:53.670379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:54:53.670379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:54:53.670379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:54:53.670379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:54:53.670379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:54:53.671149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:54:53.671149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:54:53.671149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:54:53.671149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:54:53.671149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:54:53.671149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:54:53.671149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:54:53.671149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:54:53.671149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:54:53.671149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:54:53.671149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:54:53.671149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:54:53.675741 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 61/121] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Jul 18 13:54:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:54:53.676632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45afb35d-69e0-4966-bafe-337298374e52 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:53.718205 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 61/122] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:54:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:54:53.718205 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b9ac0-4780-44db-9ecb-3edb97da5901 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:53.722374 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:53.739066 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:53.801467 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45afb35d-69e0-4966-bafe-337298374e52 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:53.823340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b9ac0-4780-44db-9ecb-3edb97da5901 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:53.837927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb7ba3-e883-4c1d-a993-0ff6c3c709f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:53.838858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb7ba3-e883-4c1d-a993-0ff6c3c709f0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:53.839153 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb7ba3-e883-4c1d-a993-0ff6c3c709f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:53.839438 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb7ba3-e883-4c1d-a993-0ff6c3c709f0 None admin] PATH_INFO: `/v3/users/2ecf077c36454bf782f946577207b833` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:53.840763 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb7ba3-e883-4c1d-a993-0ff6c3c709f0 None admin] RBAC: Authorizing `identity:get_user(user_id=2ecf077c36454bf782f946577207b833)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:53.842332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb7ba3-e883-4c1d-a993-0ff6c3c709f0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:53.844151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 62/123] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:54:53 2022] GET /identity/v3/users/2ecf077c36454bf782f946577207b833 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:53.844827 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb7ba3-e883-4c1d-a993-0ff6c3c709f0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:53.860141 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d35be3-2e02-42d4-9cfe-6ce170a6bb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:53.861157 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d35be3-2e02-42d4-9cfe-6ce170a6bb05 None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:53.861588 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d35be3-2e02-42d4-9cfe-6ce170a6bb05 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:53.861948 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d35be3-2e02-42d4-9cfe-6ce170a6bb05 None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003/users/7a10bd46534e415080cc3fb7cb29bb4d/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:53.863969 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d35be3-2e02-42d4-9cfe-6ce170a6bb05 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec42cd390d9e4d07bb7573020b5b1003, user_id=7a10bd46534e415080cc3fb7cb29bb4d, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:53.865543 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d35be3-2e02-42d4-9cfe-6ce170a6bb05 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:53.891976 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 62/124] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:54:53 2022] PUT /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003/users/7a10bd46534e415080cc3fb7cb29bb4d/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:54:53.891976 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d35be3-2e02-42d4-9cfe-6ce170a6bb05 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:53.957563 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:53.963338 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb7ba3-e883-4c1d-a993-0ff6c3c709f0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:54.001085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fdf3a-1745-4647-90a5-c2a9f5d62327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:54.001960 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fdf3a-1745-4647-90a5-c2a9f5d62327 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:54.002256 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fdf3a-1745-4647-90a5-c2a9f5d62327 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:54.002592 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fdf3a-1745-4647-90a5-c2a9f5d62327 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:54.010864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:54.016207 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497fdf3a-1745-4647-90a5-c2a9f5d62327 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:54:54.024964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-497fdf3a-1745-4647-90a5-c2a9f5d62327 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:54:54.024964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:54:54.024964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:54:54.024964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:54:54.024964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:54:54.024964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:54:54.024964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:54:54.024964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:54:54.024964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:54:54.024964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:54:54.024964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:54:54.024964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:54:54.024964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:54:54.024964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:54:54.024964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:54:54.024964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:54:54.025889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:54:54.025889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:54:54.025889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:54:54.025889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:54:54.025889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:54:54.025889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:54:54.025889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:54:54.025889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:54:54.025889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:54:54.025889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:54:54.025889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:54:54.025889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:54:54.025889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:54:54.025889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:54:54.025889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:54:54.025889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:54:54.026652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:54:54.026652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:54:54.026652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:54:54.026652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:54:54.026652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:54:54.026652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:54:54.026652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:54:54.026652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:54:54.026652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:54:54.026652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:54:54.026652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:54:54.026652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:54:54.026652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 63/125] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:54:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:54:54.031859 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fdf3a-1745-4647-90a5-c2a9f5d62327 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:54.043172 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d35be3-2e02-42d4-9cfe-6ce170a6bb05 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:54.102587 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34a983-ee63-45a9-8d50-d4939d2daec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:54.103282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34a983-ee63-45a9-8d50-d4939d2daec8 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:54.103712 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34a983-ee63-45a9-8d50-d4939d2daec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:54.104061 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34a983-ee63-45a9-8d50-d4939d2daec8 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:54.105441 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34a983-ee63-45a9-8d50-d4939d2daec8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:54.105754 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34a983-ee63-45a9-8d50-d4939d2daec8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:54.107184 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34a983-ee63-45a9-8d50-d4939d2daec8 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:54.118826 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 63/126] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:54:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:54.119166 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34a983-ee63-45a9-8d50-d4939d2daec8 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:54.150286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fdf3a-1745-4647-90a5-c2a9f5d62327 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:54.193431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957322c-a8ab-4089-b5dc-d85cbc9699b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:54.194959 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957322c-a8ab-4089-b5dc-d85cbc9699b4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:54.195428 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957322c-a8ab-4089-b5dc-d85cbc9699b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:54.196830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957322c-a8ab-4089-b5dc-d85cbc9699b4 None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:54.198497 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c957322c-a8ab-4089-b5dc-d85cbc9699b4 None admin] RBAC: Authorizing `identity:get_project(project_id=ec42cd390d9e4d07bb7573020b5b1003)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:54.200450 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c957322c-a8ab-4089-b5dc-d85cbc9699b4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:54.203509 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 64/127] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:54:54 2022] GET /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:54.208914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957322c-a8ab-4089-b5dc-d85cbc9699b4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:54.239353 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:54.252747 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34a983-ee63-45a9-8d50-d4939d2daec8 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:54.300133 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe924f5-82c7-48c1-a633-2123245a0f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:54.301566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe924f5-82c7-48c1-a633-2123245a0f8d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:54.302243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe924f5-82c7-48c1-a633-2123245a0f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:54.303090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe924f5-82c7-48c1-a633-2123245a0f8d None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:54.304524 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe924f5-82c7-48c1-a633-2123245a0f8d None admin] RBAC: Authorizing `identity:get_project(project_id=ec42cd390d9e4d07bb7573020b5b1003)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:54.306753 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe924f5-82c7-48c1-a633-2123245a0f8d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:54.309237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 64/128] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:54:54 2022] GET /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:54.310838 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe924f5-82c7-48c1-a633-2123245a0f8d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:54.324180 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957322c-a8ab-4089-b5dc-d85cbc9699b4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:54.330800 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:54.362125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4008ff1-6586-4921-af75-1d30fa209709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:54.362820 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4008ff1-6586-4921-af75-1d30fa209709 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:54.363116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4008ff1-6586-4921-af75-1d30fa209709 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:54.363411 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4008ff1-6586-4921-af75-1d30fa209709 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:54.364225 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4008ff1-6586-4921-af75-1d30fa209709 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:54.364547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4008ff1-6586-4921-af75-1d30fa209709 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:54.365967 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4008ff1-6586-4921-af75-1d30fa209709 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:54.377689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 65/129] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:54:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:54.378667 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4008ff1-6586-4921-af75-1d30fa209709 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:54.439235 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe924f5-82c7-48c1-a633-2123245a0f8d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:54.476211 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f73cc-4d9b-45a1-b30c-77dd2d845708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:54.477019 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f73cc-4d9b-45a1-b30c-77dd2d845708 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:54.477851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f73cc-4d9b-45a1-b30c-77dd2d845708 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:54.478249 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f73cc-4d9b-45a1-b30c-77dd2d845708 None admin] PATH_INFO: `/v3/users/2ecf077c36454bf782f946577207b833` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:54.479590 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f73cc-4d9b-45a1-b30c-77dd2d845708 None admin] RBAC: Authorizing `identity:get_user(user_id=2ecf077c36454bf782f946577207b833)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:54.481121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f73cc-4d9b-45a1-b30c-77dd2d845708 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:54.482981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 65/130] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:54:54 2022] GET /identity/v3/users/2ecf077c36454bf782f946577207b833 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:54.483799 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f73cc-4d9b-45a1-b30c-77dd2d845708 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:54.490415 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:54.505699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4008ff1-6586-4921-af75-1d30fa209709 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:54.551481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799fed73-3e6f-45aa-87d1-567bcd37c96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:54.552464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799fed73-3e6f-45aa-87d1-567bcd37c96a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:54.552771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799fed73-3e6f-45aa-87d1-567bcd37c96a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:54.553472 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799fed73-3e6f-45aa-87d1-567bcd37c96a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:54.554612 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799fed73-3e6f-45aa-87d1-567bcd37c96a None admin] RBAC: Adding query filter params (role.id=fd276ad89f2d4192b4f90e126d9362d3, scope.project.id=ec42cd390d9e4d07bb7573020b5b1003, user.id=f2f07f4728f74a2fb27ea83e4cc23351) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:54.554942 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799fed73-3e6f-45aa-87d1-567bcd37c96a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:54.557324 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799fed73-3e6f-45aa-87d1-567bcd37c96a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:54.565198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 66/131] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:54:54 2022] GET /identity/v3/role_assignments?user.id=f2f07f4728f74a2fb27ea83e4cc23351&scope.project.id=ec42cd390d9e4d07bb7573020b5b1003&role.id=fd276ad89f2d4192b4f90e126d9362d3 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:54.566003 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799fed73-3e6f-45aa-87d1-567bcd37c96a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:54.572285 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:54.608322 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f73cc-4d9b-45a1-b30c-77dd2d845708 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:54.654542 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f231b6f-4be8-42e5-9f4f-bca4456e38d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:54.655902 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f231b6f-4be8-42e5-9f4f-bca4456e38d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:54.656479 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f231b6f-4be8-42e5-9f4f-bca4456e38d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:54.657165 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f231b6f-4be8-42e5-9f4f-bca4456e38d2 None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003/users/2ecf077c36454bf782f946577207b833/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:54.660055 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f231b6f-4be8-42e5-9f4f-bca4456e38d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec42cd390d9e4d07bb7573020b5b1003, user_id=2ecf077c36454bf782f946577207b833, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:54.661856 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f231b6f-4be8-42e5-9f4f-bca4456e38d2 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:54.702353 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 66/132] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:54:54 2022] PUT /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003/users/2ecf077c36454bf782f946577207b833/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:54:54.703889 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f231b6f-4be8-42e5-9f4f-bca4456e38d2 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:54.753758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799fed73-3e6f-45aa-87d1-567bcd37c96a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:54.835799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51286ef-8d0a-4f16-9413-80a443338c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:54.839222 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51286ef-8d0a-4f16-9413-80a443338c2d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:54.839785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51286ef-8d0a-4f16-9413-80a443338c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:54.841344 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51286ef-8d0a-4f16-9413-80a443338c2d None admin] PATH_INFO: `/v3/users/2ecf077c36454bf782f946577207b833` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:54.846512 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51286ef-8d0a-4f16-9413-80a443338c2d None admin] RBAC: Authorizing `identity:get_user(user_id=2ecf077c36454bf782f946577207b833)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:54.848396 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51286ef-8d0a-4f16-9413-80a443338c2d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:54.852276 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 67/133] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:54:54 2022] GET /identity/v3/users/2ecf077c36454bf782f946577207b833 => generated 275 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:54.858552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069938d1-18c4-41bd-9989-13393241238a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:54.859151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069938d1-18c4-41bd-9989-13393241238a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:54.860561 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069938d1-18c4-41bd-9989-13393241238a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:54.862870 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 68/134] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:54.870721 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069938d1-18c4-41bd-9989-13393241238a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:54.924498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:54.957053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f231b6f-4be8-42e5-9f4f-bca4456e38d2 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:55.004743 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7a5db-89ae-4996-9403-c5ef4e85d73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:55.006016 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7a5db-89ae-4996-9403-c5ef4e85d73e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:55.006559 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7a5db-89ae-4996-9403-c5ef4e85d73e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:55.007050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7a5db-89ae-4996-9403-c5ef4e85d73e None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:55.009080 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7a5db-89ae-4996-9403-c5ef4e85d73e None admin] RBAC: Authorizing `identity:get_project(project_id=ec42cd390d9e4d07bb7573020b5b1003)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:55.010771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7a5db-89ae-4996-9403-c5ef4e85d73e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:55.012053 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069938d1-18c4-41bd-9989-13393241238a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:55.013049 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 67/135] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:54:54 2022] GET /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003 => generated 298 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:55.014523 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259f74b-6e4b-4314-929a-e328b1356e9f None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:55.015025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259f74b-6e4b-4314-929a-e328b1356e9f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:55.015505 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259f74b-6e4b-4314-929a-e328b1356e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:55.036879 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:55.049447 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015fbf3f-117a-4001-91a5-664ea7d9ff35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:55.050534 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fbf3f-117a-4001-91a5-664ea7d9ff35 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:55.050833 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fbf3f-117a-4001-91a5-664ea7d9ff35 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:55.051116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fbf3f-117a-4001-91a5-664ea7d9ff35 None admin] PATH_INFO: `/v3/projects/d045f37f21894bc083e89f3fed0768d5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:55.051906 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-6259f74b-6e4b-4314-929a-e328b1356e9f None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:55.052692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015fbf3f-117a-4001-91a5-664ea7d9ff35 None admin] RBAC: Authorizing `identity:get_project(project_id=d045f37f21894bc083e89f3fed0768d5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:55.054250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6259f74b-6e4b-4314-929a-e328b1356e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:55.054356 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015fbf3f-117a-4001-91a5-664ea7d9ff35 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:55.056122 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 69/136] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:54:54 2022] GET /identity/v3/projects/d045f37f21894bc083e89f3fed0768d5 => generated 312 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:55.057146 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015fbf3f-117a-4001-91a5-664ea7d9ff35 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:55.166553 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015fbf3f-117a-4001-91a5-664ea7d9ff35 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:55.176965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 68/137] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:54:55.177476 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:55.178071 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259f74b-6e4b-4314-929a-e328b1356e9f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:55.208415 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393515d-8bef-4a10-80bd-a5bc965796c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:55.209412 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393515d-8bef-4a10-80bd-a5bc965796c9 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:55.209709 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393515d-8bef-4a10-80bd-a5bc965796c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:55.210051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393515d-8bef-4a10-80bd-a5bc965796c9 None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003/users/2ecf077c36454bf782f946577207b833/roles/580aae575f414b3c9b4ccf1e27e0b640` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:55.220856 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393515d-8bef-4a10-80bd-a5bc965796c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec42cd390d9e4d07bb7573020b5b1003, user_id=2ecf077c36454bf782f946577207b833, role_id=580aae575f414b3c9b4ccf1e27e0b640)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:55.222474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393515d-8bef-4a10-80bd-a5bc965796c9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:55.246742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 70/138] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:54:55 2022] PUT /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003/users/2ecf077c36454bf782f946577207b833/roles/580aae575f414b3c9b4ccf1e27e0b640 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:54:55.250449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393515d-8bef-4a10-80bd-a5bc965796c9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:55.345750 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259f74b-6e4b-4314-929a-e328b1356e9f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:55.418634 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbd1b3-b58b-4f10-87ab-4e7733d30f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:55.419618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbd1b3-b58b-4f10-87ab-4e7733d30f31 None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:55.419913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbd1b3-b58b-4f10-87ab-4e7733d30f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:55.420218 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbd1b3-b58b-4f10-87ab-4e7733d30f31 None admin] PATH_INFO: `/v3/projects/d045f37f21894bc083e89f3fed0768d5/users/2ecf077c36454bf782f946577207b833/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:55.422866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393515d-8bef-4a10-80bd-a5bc965796c9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:55.426482 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dbd1b3-b58b-4f10-87ab-4e7733d30f31 None admin] RBAC: Authorizing `identity:create_grant(project_id=d045f37f21894bc083e89f3fed0768d5, user_id=2ecf077c36454bf782f946577207b833, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:55.427988 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dbd1b3-b58b-4f10-87ab-4e7733d30f31 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:55.441601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:55.464520 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 69/139] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:54:55 2022] PUT /identity/v3/projects/d045f37f21894bc083e89f3fed0768d5/users/2ecf077c36454bf782f946577207b833/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:54:55.487633 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866ba78-8a5b-4c5f-b1f7-30030452b3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:55.488496 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866ba78-8a5b-4c5f-b1f7-30030452b3cd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:55.488793 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866ba78-8a5b-4c5f-b1f7-30030452b3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:55.489084 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866ba78-8a5b-4c5f-b1f7-30030452b3cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:55.493426 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866ba78-8a5b-4c5f-b1f7-30030452b3cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:55.494967 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866ba78-8a5b-4c5f-b1f7-30030452b3cd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:55.496954 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 71/140] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:54:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:55.502645 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbd1b3-b58b-4f10-87ab-4e7733d30f31 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:55.634499 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbd1b3-b58b-4f10-87ab-4e7733d30f31 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:55.679969 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945bc00-4b25-4fb4-8daa-c9ee7696a673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:55.681227 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945bc00-4b25-4fb4-8daa-c9ee7696a673 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:55.681526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945bc00-4b25-4fb4-8daa-c9ee7696a673 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:55.681811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945bc00-4b25-4fb4-8daa-c9ee7696a673 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:55.682607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9945bc00-4b25-4fb4-8daa-c9ee7696a673 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:55.684104 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9945bc00-4b25-4fb4-8daa-c9ee7696a673 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:55.720458 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 70/141] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Jul 18 13:54:55 2022] POST /identity/v3/users => generated 283 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:54:56.172242 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0648e-79f3-4fc8-b413-d03ba7c28dd3 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:56.172741 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0648e-79f3-4fc8-b413-d03ba7c28dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:56.172841 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0648e-79f3-4fc8-b413-d03ba7c28dd3 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:56.174198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 72/142] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:56.179582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b306e6-25ea-40a6-bb05-4eb4733fc52d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:56.179887 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b306e6-25ea-40a6-bb05-4eb4733fc52d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:56.180224 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b306e6-25ea-40a6-bb05-4eb4733fc52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:56.205029 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-12b306e6-25ea-40a6-bb05-4eb4733fc52d None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:56.206157 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-12b306e6-25ea-40a6-bb05-4eb4733fc52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:56.259515 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:56.286271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 71/143] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:54:56.323320 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0648e-79f3-4fc8-b413-d03ba7c28dd3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:56.343392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:56.453916 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0648e-79f3-4fc8-b413-d03ba7c28dd3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:56.499248 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9ed05-4df5-4854-bcb5-d9902ff81477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:56.500319 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9ed05-4df5-4854-bcb5-d9902ff81477 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:56.500756 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9ed05-4df5-4854-bcb5-d9902ff81477 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:56.501180 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9ed05-4df5-4854-bcb5-d9902ff81477 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:56.522748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9ed05-4df5-4854-bcb5-d9902ff81477 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:54:56.531209 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-06c9ed05-4df5-4854-bcb5-d9902ff81477 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:54:56.531209 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:54:56.531209 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:54:56.531209 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:54:56.531209 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:54:56.531209 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:54:56.531209 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:54:56.531209 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:54:56.531209 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:54:56.531209 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:54:56.531209 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:54:56.531209 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:54:56.531209 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:54:56.531209 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:54:56.531209 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:54:56.531209 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:54:56.532274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:54:56.532274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:54:56.532274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:54:56.532274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:54:56.532274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:54:56.532274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:54:56.532274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:54:56.532274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:54:56.532274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:54:56.532274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:54:56.532274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:54:56.532274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:54:56.532274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:54:56.532274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:54:56.532274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:54:56.532274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:54:56.533237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:54:56.533237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:54:56.533237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:54:56.533237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:54:56.533237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:54:56.533237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:54:56.533237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:54:56.533237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:54:56.533237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:54:56.533237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:54:56.533237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:54:56.533237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:54:56.535243 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 73/144] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:54:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:54:56.542012 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b306e6-25ea-40a6-bb05-4eb4733fc52d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:56.675296 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b306e6-25ea-40a6-bb05-4eb4733fc52d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:56.716502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20006ea6-edba-4598-8a44-97a223a53575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:56.717185 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20006ea6-edba-4598-8a44-97a223a53575 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:56.717469 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20006ea6-edba-4598-8a44-97a223a53575 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:56.717745 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20006ea6-edba-4598-8a44-97a223a53575 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:56.718497 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20006ea6-edba-4598-8a44-97a223a53575 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:56.718800 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20006ea6-edba-4598-8a44-97a223a53575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:56.720275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20006ea6-edba-4598-8a44-97a223a53575 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:56.729915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 72/145] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:54:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:56.735472 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9ed05-4df5-4854-bcb5-d9902ff81477 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:56.838939 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:56.867935 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9ed05-4df5-4854-bcb5-d9902ff81477 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:56.914000 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944f210-9ee1-407c-9119-e3d06ae61a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:56.914920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944f210-9ee1-407c-9119-e3d06ae61a81 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:56.915214 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944f210-9ee1-407c-9119-e3d06ae61a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:56.915498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944f210-9ee1-407c-9119-e3d06ae61a81 None admin] PATH_INFO: `/v3/users/7a10bd46534e415080cc3fb7cb29bb4d` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:56.918508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944f210-9ee1-407c-9119-e3d06ae61a81 None admin] RBAC: Authorizing `identity:get_user(user_id=7a10bd46534e415080cc3fb7cb29bb4d)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:56.920009 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944f210-9ee1-407c-9119-e3d06ae61a81 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:56.921673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 74/146] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:54:56 2022] GET /identity/v3/users/7a10bd46534e415080cc3fb7cb29bb4d => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:56.927386 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20006ea6-edba-4598-8a44-97a223a53575 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:57.011080 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:57.072977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20006ea6-edba-4598-8a44-97a223a53575 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:57.117245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7db84-e1fc-4080-8728-ded06bebe40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:57.117833 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7db84-e1fc-4080-8728-ded06bebe40d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:57.118660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7db84-e1fc-4080-8728-ded06bebe40d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:57.118942 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7db84-e1fc-4080-8728-ded06bebe40d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:57.200864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7db84-e1fc-4080-8728-ded06bebe40d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:57.202472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7db84-e1fc-4080-8728-ded06bebe40d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:57.261398 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:57.330930 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 73/147] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:54:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:54:57.336469 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944f210-9ee1-407c-9119-e3d06ae61a81 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:57.471085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944f210-9ee1-407c-9119-e3d06ae61a81 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:57.484407 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999f86c-050b-499a-ad27-d6ba5feef7c8 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:57.484723 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999f86c-050b-499a-ad27-d6ba5feef7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:57.484981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999f86c-050b-499a-ad27-d6ba5feef7c8 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:57.486500 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 74/148] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:57.494737 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f9ed9-2e89-4fc5-a8c0-174bd0436ced None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:57.495029 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f9ed9-2e89-4fc5-a8c0-174bd0436ced None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:57.495370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f9ed9-2e89-4fc5-a8c0-174bd0436ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:57.509925 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28967c9e-4810-49c9-b34c-475b41dee977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:57.510888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28967c9e-4810-49c9-b34c-475b41dee977 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:57.511182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28967c9e-4810-49c9-b34c-475b41dee977 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:57.511469 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28967c9e-4810-49c9-b34c-475b41dee977 None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:57.515594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28967c9e-4810-49c9-b34c-475b41dee977 None admin] RBAC: Authorizing `identity:get_project(project_id=ec42cd390d9e4d07bb7573020b5b1003)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:57.517123 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28967c9e-4810-49c9-b34c-475b41dee977 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:57.519057 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 75/149] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:54:57 2022] GET /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:57.525092 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-db7f9ed9-2e89-4fc5-a8c0-174bd0436ced None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:57.526346 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-db7f9ed9-2e89-4fc5-a8c0-174bd0436ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:57.533445 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28967c9e-4810-49c9-b34c-475b41dee977 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:57.554485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:57.671895 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 75/150] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:54:57.674167 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28967c9e-4810-49c9-b34c-475b41dee977 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:57.716653 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7e9e4-ede3-41cf-b53e-ea74321eb4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:57.717337 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7e9e4-ede3-41cf-b53e-ea74321eb4fb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:57.717633 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7e9e4-ede3-41cf-b53e-ea74321eb4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:57.717921 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7e9e4-ede3-41cf-b53e-ea74321eb4fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:57.718766 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7e9e4-ede3-41cf-b53e-ea74321eb4fb None admin] RBAC: Adding query filter params (role.id=c60a99de086f45c6820f7872a6b43feb, scope.project.id=ec42cd390d9e4d07bb7573020b5b1003, user.id=7a10bd46534e415080cc3fb7cb29bb4d) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:57.719087 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7e9e4-ede3-41cf-b53e-ea74321eb4fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:57.720599 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7e9e4-ede3-41cf-b53e-ea74321eb4fb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:57.727026 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 76/151] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:54:57 2022] GET /identity/v3/role_assignments?user.id=7a10bd46534e415080cc3fb7cb29bb4d&scope.project.id=ec42cd390d9e4d07bb7573020b5b1003&role.id=c60a99de086f45c6820f7872a6b43feb => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:54:57.732218 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f9ed9-2e89-4fc5-a8c0-174bd0436ced None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:57.831192 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:57.875857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f9ed9-2e89-4fc5-a8c0-174bd0436ced None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:57.916333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f9ec49-3f88-4361-833f-a547b841ad34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:57.917585 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9ec49-3f88-4361-833f-a547b841ad34 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:57.918095 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9ec49-3f88-4361-833f-a547b841ad34 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:57.918585 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9ec49-3f88-4361-833f-a547b841ad34 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:57.919158 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56f969-9352-439e-9156-20069e0284f0 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:57.919473 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56f969-9352-439e-9156-20069e0284f0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:57.919752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56f969-9352-439e-9156-20069e0284f0 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:57.921179 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 77/152] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:54:57.926491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7faf033-325e-4101-9fa2-afe7fa7072da None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:57.926792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7faf033-325e-4101-9fa2-afe7fa7072da None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:57.927082 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7faf033-325e-4101-9fa2-afe7fa7072da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:57.956773 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f9ec49-3f88-4361-833f-a547b841ad34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:54:57.964993 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b7faf033-325e-4101-9fa2-afe7fa7072da None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:57.965367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-d8f9ec49-3f88-4361-833f-a547b841ad34 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 13:54:57.965367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:54:57.965367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:54:57.965367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:54:57.965367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:54:57.965367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:54:57.965367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:54:57.965367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:54:57.965367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:54:57.965367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:54:57.965367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:54:57.965367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:54:57.965367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:54:57.965367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:54:57.965367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:54:57.965367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:54:57.966223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:54:57.966223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:54:57.966223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:54:57.966223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:54:57.966223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:54:57.966223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:54:57.966223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:54:57.966223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:54:57.966223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:54:57.966223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:54:57.966223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:54:57.966223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:54:57.966223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:54:57.966223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:54:57.966223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:54:57.966223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:54:57.967003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:54:57.967003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:54:57.967003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:54:57.967003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:54:57.967003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:54:57.967003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:54:57.967003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:54:57.967003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:54:57.967003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:54:57.967003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:54:57.967003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 13:54:57.967003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:54:57.967631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b7faf033-325e-4101-9fa2-afe7fa7072da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:57.969962 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 76/153] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:54:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:54:57.994603 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f9ec49-3f88-4361-833f-a547b841ad34 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:58.087967 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 78/154] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 18 13:54:58.126272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7faf033-325e-4101-9fa2-afe7fa7072da None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:58.138676 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:58.204167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f9ec49-3f88-4361-833f-a547b841ad34 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:58.277645 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa4079-8e7f-46c2-8a55-bf495737e27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:58.278247 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa4079-8e7f-46c2-8a55-bf495737e27b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:58.278535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa4079-8e7f-46c2-8a55-bf495737e27b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:58.278813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa4079-8e7f-46c2-8a55-bf495737e27b None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:58.279565 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa4079-8e7f-46c2-8a55-bf495737e27b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:58.279872 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa4079-8e7f-46c2-8a55-bf495737e27b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:58.281281 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa4079-8e7f-46c2-8a55-bf495737e27b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:58.284230 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:58.299488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 77/155] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:54:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:54:58.305125 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa4079-8e7f-46c2-8a55-bf495737e27b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:58.411699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7faf033-325e-4101-9fa2-afe7fa7072da None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:58.491895 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c6f13-f5be-4b78-84a2-99f83a47d350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:58.492738 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c6f13-f5be-4b78-84a2-99f83a47d350 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:58.493557 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c6f13-f5be-4b78-84a2-99f83a47d350 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:58.493841 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c6f13-f5be-4b78-84a2-99f83a47d350 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:58.520866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c6f13-f5be-4b78-84a2-99f83a47d350 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:54:58.529250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-695c6f13-f5be-4b78-84a2-99f83a47d350 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:54:58.529250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:54:58.529250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:54:58.529250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:54:58.529250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:54:58.529250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:54:58.529250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:54:58.529250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:54:58.529250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:54:58.529250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:54:58.529250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:54:58.529250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:54:58.529250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:54:58.529250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:54:58.529250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:54:58.529250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:54:58.530306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:54:58.530306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:54:58.530306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:54:58.530306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:54:58.530306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:54:58.530306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:54:58.530306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:54:58.530306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:54:58.530306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:54:58.530306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:54:58.530306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:54:58.530306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:54:58.530306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:54:58.530306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:54:58.530306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:54:58.530306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:54:58.531155 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:54:58.531155 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:54:58.531155 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:54:58.531155 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:54:58.531155 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:54:58.531155 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:54:58.531155 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:54:58.531155 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:54:58.531155 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:54:58.531155 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:54:58.531155 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:54:58.531155 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:54:58.535311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 79/156] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:54:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:54:58.536183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c6f13-f5be-4b78-84a2-99f83a47d350 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:58.553905 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa4079-8e7f-46c2-8a55-bf495737e27b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:58.638146 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4663a-e3a2-42b5-a085-7883fd93c031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:58.639039 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4663a-e3a2-42b5-a085-7883fd93c031 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:58.639322 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4663a-e3a2-42b5-a085-7883fd93c031 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:58.639602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4663a-e3a2-42b5-a085-7883fd93c031 None admin] PATH_INFO: `/v3/users/f2f07f4728f74a2fb27ea83e4cc23351` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:58.644882 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4663a-e3a2-42b5-a085-7883fd93c031 None admin] RBAC: Authorizing `identity:get_user(user_id=f2f07f4728f74a2fb27ea83e4cc23351)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:58.646368 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4663a-e3a2-42b5-a085-7883fd93c031 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:58.648402 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 78/157] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:54:58 2022] GET /identity/v3/users/f2f07f4728f74a2fb27ea83e4cc23351 => generated 289 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:54:58.664997 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4663a-e3a2-42b5-a085-7883fd93c031 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:58.668279 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:58.716342 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c6f13-f5be-4b78-84a2-99f83a47d350 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:58.763570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:58.781671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea2cdf-b651-4547-899e-1a6af548415e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:58.782279 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea2cdf-b651-4547-899e-1a6af548415e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:58.782571 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea2cdf-b651-4547-899e-1a6af548415e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:58.782858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea2cdf-b651-4547-899e-1a6af548415e None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:58.783622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea2cdf-b651-4547-899e-1a6af548415e None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:58.783937 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea2cdf-b651-4547-899e-1a6af548415e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:58.785362 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea2cdf-b651-4547-899e-1a6af548415e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:58.802804 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 80/158] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:54:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:58.808227 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea2cdf-b651-4547-899e-1a6af548415e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:58.825829 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4663a-e3a2-42b5-a085-7883fd93c031 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:58.891105 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741410a-1890-4261-9858-30dea366779c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:58.892378 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741410a-1890-4261-9858-30dea366779c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:58.892878 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741410a-1890-4261-9858-30dea366779c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:58.893463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741410a-1890-4261-9858-30dea366779c None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:58.894967 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741410a-1890-4261-9858-30dea366779c None admin] RBAC: Authorizing `identity:get_project(project_id=ec42cd390d9e4d07bb7573020b5b1003)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:58.903289 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741410a-1890-4261-9858-30dea366779c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:58.905490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 79/159] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:54:58 2022] GET /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003 => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:54:58.911904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c3833-e1a4-46fd-a142-19341725d3de None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:58.912465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c3833-e1a4-46fd-a142-19341725d3de None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:58.913132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c3833-e1a4-46fd-a142-19341725d3de None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:58.915412 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 80/160] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:58.916737 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40824619-6982-40fd-9d04-4ae40097748b None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:58.917242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40824619-6982-40fd-9d04-4ae40097748b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:58.917815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40824619-6982-40fd-9d04-4ae40097748b None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:58.919339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 81/161] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:54:58.924832 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbc148-5835-4732-8b47-3e887d1c93d2 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:58.925123 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbc148-5835-4732-8b47-3e887d1c93d2 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:58.925458 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbc148-5835-4732-8b47-3e887d1c93d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:58.957116 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c7bbc148-5835-4732-8b47-3e887d1c93d2 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:58.958334 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c7bbc148-5835-4732-8b47-3e887d1c93d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:58.962520 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea2cdf-b651-4547-899e-1a6af548415e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:59.012285 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:59.024369 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c2513-14c7-4c8a-bd52-4bc6c11390a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:59.025336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c2513-14c7-4c8a-bd52-4bc6c11390a1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:59.025631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c2513-14c7-4c8a-bd52-4bc6c11390a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:59.025917 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c2513-14c7-4c8a-bd52-4bc6c11390a1 None admin] PATH_INFO: `/v3/users/2ecf077c36454bf782f946577207b833` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:59.027631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c2513-14c7-4c8a-bd52-4bc6c11390a1 None admin] RBAC: Authorizing `identity:get_user(user_id=2ecf077c36454bf782f946577207b833)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:59.029163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c2513-14c7-4c8a-bd52-4bc6c11390a1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:59.035883 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 81/162] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:54:58 2022] GET /identity/v3/users/2ecf077c36454bf782f946577207b833 => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:59.037375 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87423ef-3b4e-45ce-bdba-b470aa8b31a3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:59.037879 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87423ef-3b4e-45ce-bdba-b470aa8b31a3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:59.038373 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87423ef-3b4e-45ce-bdba-b470aa8b31a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:59.058647 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c87423ef-3b4e-45ce-bdba-b470aa8b31a3 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:54:59.061389 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c87423ef-3b4e-45ce-bdba-b470aa8b31a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:54:59.105079 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 82/163] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:54:59.107494 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbc148-5835-4732-8b47-3e887d1c93d2 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:59.190238 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 82/164] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:54:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:54:59.195964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87423ef-3b4e-45ce-bdba-b470aa8b31a3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:59.209234 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:59.263710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbc148-5835-4732-8b47-3e887d1c93d2 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:59.325289 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a07dde-0c14-4c1f-8cdb-dd47db9b7812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:59.326116 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a07dde-0c14-4c1f-8cdb-dd47db9b7812 None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:59.326402 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a07dde-0c14-4c1f-8cdb-dd47db9b7812 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:59.326677 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a07dde-0c14-4c1f-8cdb-dd47db9b7812 None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003/users/f2f07f4728f74a2fb27ea83e4cc23351/roles/fd276ad89f2d4192b4f90e126d9362d3` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:59.333275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a07dde-0c14-4c1f-8cdb-dd47db9b7812 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec42cd390d9e4d07bb7573020b5b1003, user_id=f2f07f4728f74a2fb27ea83e4cc23351, role_id=fd276ad89f2d4192b4f90e126d9362d3)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:59.334738 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a07dde-0c14-4c1f-8cdb-dd47db9b7812 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:59.348329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87423ef-3b4e-45ce-bdba-b470aa8b31a3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:59.366319 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 83/165] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:54:59 2022] PUT /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003/users/f2f07f4728f74a2fb27ea83e4cc23351/roles/fd276ad89f2d4192b4f90e126d9362d3 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:54:59.367073 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a07dde-0c14-4c1f-8cdb-dd47db9b7812 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:59.378869 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:59.389106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0f237-efba-4d12-9886-522c216531f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:59.390316 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0f237-efba-4d12-9886-522c216531f7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:59.390875 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0f237-efba-4d12-9886-522c216531f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:59.391511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0f237-efba-4d12-9886-522c216531f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:59.470563 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0f237-efba-4d12-9886-522c216531f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:59.472927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0f237-efba-4d12-9886-522c216531f7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:59.549209 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a07dde-0c14-4c1f-8cdb-dd47db9b7812 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:59.580560 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 83/166] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:54:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:54:59.581720 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0f237-efba-4d12-9886-522c216531f7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:59.593666 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:59.618731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:59.620287 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f53e26-7064-4165-a4ee-813e96b64420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:59.621066 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f53e26-7064-4165-a4ee-813e96b64420 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:59.621353 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f53e26-7064-4165-a4ee-813e96b64420 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:59.621697 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f53e26-7064-4165-a4ee-813e96b64420 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:59.648862 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f53e26-7064-4165-a4ee-813e96b64420 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:54:59.657273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-01f53e26-7064-4165-a4ee-813e96b64420 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:54:59.657273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:54:59.657273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:54:59.657273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:54:59.657273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:54:59.657273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:54:59.657273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:54:59.657273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:54:59.657273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:54:59.657273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:54:59.657273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:54:59.657273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:54:59.657273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:54:59.657273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:54:59.657273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:54:59.657273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:54:59.658222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:54:59.658222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:54:59.658222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:54:59.658222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:54:59.658222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:54:59.658222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:54:59.658222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:54:59.658222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:54:59.658222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:54:59.658222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:54:59.658222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:54:59.658222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:54:59.658222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:54:59.658222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:54:59.658222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:54:59.658222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:54:59.659256 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:54:59.659256 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:54:59.659256 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:54:59.659256 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:54:59.659256 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:54:59.659256 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:54:59.659256 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:54:59.659256 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:54:59.659256 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:54:59.659256 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:54:59.659256 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:54:59.659256 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:54:59.663516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 84/167] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:54:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:54:59.664525 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f53e26-7064-4165-a4ee-813e96b64420 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:59.702450 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0f237-efba-4d12-9886-522c216531f7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:59.738512 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e77121-a1ce-461f-afe9-6ba10b36b596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:59.739698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e77121-a1ce-461f-afe9-6ba10b36b596 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:59.740220 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e77121-a1ce-461f-afe9-6ba10b36b596 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:59.740697 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e77121-a1ce-461f-afe9-6ba10b36b596 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:59.751666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e77121-a1ce-461f-afe9-6ba10b36b596 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:54:59.761336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-42e77121-a1ce-461f-afe9-6ba10b36b596 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:54:59.761336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:54:59.761336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:54:59.761336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:54:59.761336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:54:59.761336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:54:59.761336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:54:59.761336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:54:59.761336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:54:59.761336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:54:59.761336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:54:59.761336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:54:59.761336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:54:59.761336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:54:59.761336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:54:59.761336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:54:59.762230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:54:59.762230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:54:59.762230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:54:59.762230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:54:59.762230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:54:59.762230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:54:59.762230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:54:59.762230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:54:59.762230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:54:59.762230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:54:59.762230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:54:59.762230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:54:59.762230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:54:59.762230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:54:59.762230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:54:59.762230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:54:59.763011 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:54:59.763011 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:54:59.763011 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:54:59.763011 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:54:59.763011 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:54:59.763011 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:54:59.763011 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:54:59.763011 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:54:59.763011 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:54:59.763011 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:54:59.763011 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:54:59.763011 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:54:59.765541 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 84/168] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Jul 18 13:54:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:54:59.766526 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e77121-a1ce-461f-afe9-6ba10b36b596 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:59.837569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f53e26-7064-4165-a4ee-813e96b64420 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:59.863782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:59.876970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e77121-a1ce-461f-afe9-6ba10b36b596 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:54:59.880729 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbee59-1619-45dc-80b6-f03ae30d9d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:59.881704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbee59-1619-45dc-80b6-f03ae30d9d7f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:59.882074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbee59-1619-45dc-80b6-f03ae30d9d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:59.882432 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbee59-1619-45dc-80b6-f03ae30d9d7f None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:59.884378 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fbee59-1619-45dc-80b6-f03ae30d9d7f None admin] RBAC: Authorizing `identity:get_project(project_id=ec42cd390d9e4d07bb7573020b5b1003)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:59.885932 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fbee59-1619-45dc-80b6-f03ae30d9d7f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:59.888235 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 85/169] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:54:59 2022] GET /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:59.889573 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbee59-1619-45dc-80b6-f03ae30d9d7f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:54:59.941110 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906231c1-3223-4e1c-910d-9a9f692f36dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:54:59.942326 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906231c1-3223-4e1c-910d-9a9f692f36dc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:54:59.942779 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906231c1-3223-4e1c-910d-9a9f692f36dc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:54:59.943230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906231c1-3223-4e1c-910d-9a9f692f36dc None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:54:59.944257 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906231c1-3223-4e1c-910d-9a9f692f36dc None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:54:59.944747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906231c1-3223-4e1c-910d-9a9f692f36dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:54:59.952447 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:54:59.953582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906231c1-3223-4e1c-910d-9a9f692f36dc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:54:59.968577 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 85/170] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:54:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:54:59.969452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906231c1-3223-4e1c-910d-9a9f692f36dc None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:00.017890 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbee59-1619-45dc-80b6-f03ae30d9d7f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:00.058042 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fe009-98a4-4f3f-8eba-fb3428450840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:00.058877 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fe009-98a4-4f3f-8eba-fb3428450840 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:00.059174 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fe009-98a4-4f3f-8eba-fb3428450840 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:00.059463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fe009-98a4-4f3f-8eba-fb3428450840 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:00.060302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fe009-98a4-4f3f-8eba-fb3428450840 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:00.060620 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fe009-98a4-4f3f-8eba-fb3428450840 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:00.062045 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fe009-98a4-4f3f-8eba-fb3428450840 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:00.071959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 86/171] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:54:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:00.077078 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fe009-98a4-4f3f-8eba-fb3428450840 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:00.169781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906231c1-3223-4e1c-910d-9a9f692f36dc None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:00.193582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fe009-98a4-4f3f-8eba-fb3428450840 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:00.210585 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:00.221743 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55253317-9f76-45eb-80bb-c7f8e5bc2a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:00.222951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55253317-9f76-45eb-80bb-c7f8e5bc2a7a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:00.223574 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55253317-9f76-45eb-80bb-c7f8e5bc2a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:00.224106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55253317-9f76-45eb-80bb-c7f8e5bc2a7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:00.225146 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55253317-9f76-45eb-80bb-c7f8e5bc2a7a None admin] RBAC: Adding query filter params (role.id=1399a838106b4f9bb5fa1c942cc3ef22, scope.project.id=ec42cd390d9e4d07bb7573020b5b1003, user.id=2ecf077c36454bf782f946577207b833) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:00.225763 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55253317-9f76-45eb-80bb-c7f8e5bc2a7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:00.227659 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55253317-9f76-45eb-80bb-c7f8e5bc2a7a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:00.234933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 86/172] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:54:59 2022] GET /identity/v3/role_assignments?user.id=2ecf077c36454bf782f946577207b833&scope.project.id=ec42cd390d9e4d07bb7573020b5b1003&role.id=1399a838106b4f9bb5fa1c942cc3ef22 => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:00.235199 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a1874-2654-4098-8ed7-c00261bd632f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:00.236186 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a1874-2654-4098-8ed7-c00261bd632f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:00.236477 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55253317-9f76-45eb-80bb-c7f8e5bc2a7a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:00.236599 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a1874-2654-4098-8ed7-c00261bd632f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:00.236779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a1874-2654-4098-8ed7-c00261bd632f None admin] PATH_INFO: `/v3/users/2ecf077c36454bf782f946577207b833` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:00.238126 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a1874-2654-4098-8ed7-c00261bd632f None admin] RBAC: Authorizing `identity:get_user(user_id=2ecf077c36454bf782f946577207b833)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:00.239656 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a1874-2654-4098-8ed7-c00261bd632f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:00.242579 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 87/173] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:55:00 2022] GET /identity/v3/users/2ecf077c36454bf782f946577207b833 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:00.249897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a3c83-04ee-4129-a836-8ac9227eccb0 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:00.250197 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a3c83-04ee-4129-a836-8ac9227eccb0 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:00.250477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a3c83-04ee-4129-a836-8ac9227eccb0 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:00.252294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 88/174] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:00.256763 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a3c83-04ee-4129-a836-8ac9227eccb0 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:00.271098 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:00.373768 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a3c83-04ee-4129-a836-8ac9227eccb0 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:00.384117 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55253317-9f76-45eb-80bb-c7f8e5bc2a7a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:00.426543 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0af35-e76a-4f11-8fd3-79a6971ba5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:00.427795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0af35-e76a-4f11-8fd3-79a6971ba5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:00.429201 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0af35-e76a-4f11-8fd3-79a6971ba5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:00.430367 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0af35-e76a-4f11-8fd3-79a6971ba5d8 None admin] PATH_INFO: `/v3/users/2ecf077c36454bf782f946577207b833` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:00.433436 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0af35-e76a-4f11-8fd3-79a6971ba5d8 None admin] RBAC: Authorizing `identity:get_user(user_id=2ecf077c36454bf782f946577207b833)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:00.435916 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0af35-e76a-4f11-8fd3-79a6971ba5d8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:00.440751 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 87/175] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:55:00 2022] GET /identity/v3/users/2ecf077c36454bf782f946577207b833 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:00.442836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad793f2c-085e-4bdb-8677-b7607d0d25b2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:00.443509 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad793f2c-085e-4bdb-8677-b7607d0d25b2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:00.448285 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad793f2c-085e-4bdb-8677-b7607d0d25b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:00.449176 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69aeea0-6d45-485e-a832-1111c61e801f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:00.451233 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69aeea0-6d45-485e-a832-1111c61e801f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:00.452131 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69aeea0-6d45-485e-a832-1111c61e801f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:00.453007 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69aeea0-6d45-485e-a832-1111c61e801f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:00.486930 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ad793f2c-085e-4bdb-8677-b7607d0d25b2 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:00.489604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ad793f2c-085e-4bdb-8677-b7607d0d25b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:00.554282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:00.593311 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69aeea0-6d45-485e-a832-1111c61e801f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:00.595208 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69aeea0-6d45-485e-a832-1111c61e801f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:00.632212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 88/176] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:00.633414 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad793f2c-085e-4bdb-8677-b7607d0d25b2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:00.658317 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:00.808433 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 89/177] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:00.809283 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69aeea0-6d45-485e-a832-1111c61e801f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:00.814997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad793f2c-085e-4bdb-8677-b7607d0d25b2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:00.856731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d56c9e-7775-4d47-bbc8-2aab255bb03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:00.857414 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d56c9e-7775-4d47-bbc8-2aab255bb03f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:00.857704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d56c9e-7775-4d47-bbc8-2aab255bb03f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:00.857986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d56c9e-7775-4d47-bbc8-2aab255bb03f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:00.909679 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:00.931382 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69aeea0-6d45-485e-a832-1111c61e801f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:00.931996 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d56c9e-7775-4d47-bbc8-2aab255bb03f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:00.933476 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d56c9e-7775-4d47-bbc8-2aab255bb03f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:00.945839 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:00.972009 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c41ca-1f5d-49a3-b952-156c1b724c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:00.972815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c41ca-1f5d-49a3-b952-156c1b724c5c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:00.973106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c41ca-1f5d-49a3-b952-156c1b724c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:00.973386 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c41ca-1f5d-49a3-b952-156c1b724c5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:00.983653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c41ca-1f5d-49a3-b952-156c1b724c5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:00.991325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-3c1c41ca-1f5d-49a3-b952-156c1b724c5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:55:00.991325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:00.991325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:00.991325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:00.991325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:00.991325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:00.991325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:00.991325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:00.991325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:00.991325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:00.991325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:00.991325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:00.991325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:00.991325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:00.991325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:00.991325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:00.992448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:00.992448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:00.992448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:00.992448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:00.992448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:00.992448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:00.992448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:00.992448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:00.992448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:00.992448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:00.992448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:00.992448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:00.992448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:00.992448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:00.992448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:00.992448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:00.993787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:00.993787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:00.993787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:00.993787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:00.993787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:00.993787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:00.993787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:00.993787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:00.993787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:00.993787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:00.993787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:55:00.993787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:00.993787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 90/178] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:55:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:00.994439 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157b021-68d4-4e47-bbeb-248493e5f8b3 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:00.994544 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157b021-68d4-4e47-bbeb-248493e5f8b3 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:00.994831 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157b021-68d4-4e47-bbeb-248493e5f8b3 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:00.996112 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 91/179] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:00.996811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157b021-68d4-4e47-bbeb-248493e5f8b3 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:01.059689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 89/180] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:01.061315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80a160-2f0c-4142-8a02-c7e257160252 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:01.061755 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80a160-2f0c-4142-8a02-c7e257160252 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:01.062300 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c80a160-2f0c-4142-8a02-c7e257160252 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:01.063869 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 90/181] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:01.064709 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c80a160-2f0c-4142-8a02-c7e257160252 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:01.101606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157b021-68d4-4e47-bbeb-248493e5f8b3 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:01.137862 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef9f60-257d-4541-9c9f-128128b2cd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:01.138739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef9f60-257d-4541-9c9f-128128b2cd64 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:01.139034 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef9f60-257d-4541-9c9f-128128b2cd64 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:01.139321 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef9f60-257d-4541-9c9f-128128b2cd64 None admin] PATH_INFO: `/v3/projects/d045f37f21894bc083e89f3fed0768d5` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:01.140726 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef9f60-257d-4541-9c9f-128128b2cd64 None admin] RBAC: Authorizing `identity:get_project(project_id=d045f37f21894bc083e89f3fed0768d5)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:01.142218 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef9f60-257d-4541-9c9f-128128b2cd64 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:01.144004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 92/182] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:00 2022] GET /identity/v3/projects/d045f37f21894bc083e89f3fed0768d5 => generated 312 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:01.145241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fdf15-399a-441f-bf66-7c1a6adc62d0 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:01.145534 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fdf15-399a-441f-bf66-7c1a6adc62d0 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:01.145818 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fdf15-399a-441f-bf66-7c1a6adc62d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:01.165755 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-7a4fdf15-399a-441f-bf66-7c1a6adc62d0 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:01.166952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7a4fdf15-399a-441f-bf66-7c1a6adc62d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:01.169115 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:01.181938 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c80a160-2f0c-4142-8a02-c7e257160252 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:01.193904 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:01.219779 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775ce9d-6480-47ca-bf77-d5b226284157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:01.220593 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775ce9d-6480-47ca-bf77-d5b226284157 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:01.220913 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775ce9d-6480-47ca-bf77-d5b226284157 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:01.221235 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775ce9d-6480-47ca-bf77-d5b226284157 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:01.221919 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775ce9d-6480-47ca-bf77-d5b226284157 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:01.222223 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775ce9d-6480-47ca-bf77-d5b226284157 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:01.223617 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775ce9d-6480-47ca-bf77-d5b226284157 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:01.235980 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 91/183] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:01.235980 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775ce9d-6480-47ca-bf77-d5b226284157 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:01.245056 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 93/184] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:01.245056 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f187f0b-e1ef-4a2f-a0fc-30ed01faead3 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:01.245056 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f187f0b-e1ef-4a2f-a0fc-30ed01faead3 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:01.245056 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f187f0b-e1ef-4a2f-a0fc-30ed01faead3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:01.266478 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-3f187f0b-e1ef-4a2f-a0fc-30ed01faead3 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:01.267644 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3f187f0b-e1ef-4a2f-a0fc-30ed01faead3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:01.337610 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 94/185] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:01.348806 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f187f0b-e1ef-4a2f-a0fc-30ed01faead3 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:01.353950 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775ce9d-6480-47ca-bf77-d5b226284157 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:01.390034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77e840-5f39-466c-8669-36317b94a70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:01.390904 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77e840-5f39-466c-8669-36317b94a70e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:01.391193 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77e840-5f39-466c-8669-36317b94a70e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:01.391467 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77e840-5f39-466c-8669-36317b94a70e None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:01.392692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a77e840-5f39-466c-8669-36317b94a70e None admin] RBAC: Authorizing `identity:get_project(project_id=ec42cd390d9e4d07bb7573020b5b1003)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:01.394158 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a77e840-5f39-466c-8669-36317b94a70e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:01.396003 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 92/186] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:01 2022] GET /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:01.396633 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77e840-5f39-466c-8669-36317b94a70e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:01.432335 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:01.440484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:01.468261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f187f0b-e1ef-4a2f-a0fc-30ed01faead3 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:01.502429 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77e840-5f39-466c-8669-36317b94a70e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:01.506074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4fd6a4-e16b-4762-bbf4-ad71d7d8cf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:01.506739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4fd6a4-e16b-4762-bbf4-ad71d7d8cf3d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:01.507027 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4fd6a4-e16b-4762-bbf4-ad71d7d8cf3d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:01.507310 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4fd6a4-e16b-4762-bbf4-ad71d7d8cf3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:01.508173 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4fd6a4-e16b-4762-bbf4-ad71d7d8cf3d None admin] RBAC: Adding query filter params (role.id=1399a838106b4f9bb5fa1c942cc3ef22, scope.project.id=d045f37f21894bc083e89f3fed0768d5, user.id=2ecf077c36454bf782f946577207b833) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:01.508491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4fd6a4-e16b-4762-bbf4-ad71d7d8cf3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:01.510109 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4fd6a4-e16b-4762-bbf4-ad71d7d8cf3d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:01.516303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 95/187] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:55:01 2022] GET /identity/v3/role_assignments?user.id=2ecf077c36454bf782f946577207b833&scope.project.id=d045f37f21894bc083e89f3fed0768d5&role.id=1399a838106b4f9bb5fa1c942cc3ef22 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:01.516989 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4fd6a4-e16b-4762-bbf4-ad71d7d8cf3d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:01.544373 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fdff3-76c2-461e-b3fb-469b7dc38533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:01.545985 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fdff3-76c2-461e-b3fb-469b7dc38533 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:01.546277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fdff3-76c2-461e-b3fb-469b7dc38533 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:01.546566 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fdff3-76c2-461e-b3fb-469b7dc38533 None admin] PATH_INFO: `/v3/users/49eedf7e18eb4505889fa1b78da2b449` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:01.568303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328fdff3-76c2-461e-b3fb-469b7dc38533 None admin] RBAC: Authorizing `identity:get_user(user_id=49eedf7e18eb4505889fa1b78da2b449)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:01.570628 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328fdff3-76c2-461e-b3fb-469b7dc38533 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:01.573930 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 93/188] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:55:01 2022] GET /identity/v3/users/49eedf7e18eb4505889fa1b78da2b449 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:01.580738 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fdff3-76c2-461e-b3fb-469b7dc38533 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:01.635197 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4fd6a4-e16b-4762-bbf4-ad71d7d8cf3d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:01.667722 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:01.674217 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b06538-75d6-44a8-9b8e-1e39c8f186df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:01.675038 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b06538-75d6-44a8-9b8e-1e39c8f186df None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:01.675335 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b06538-75d6-44a8-9b8e-1e39c8f186df None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:01.675621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b06538-75d6-44a8-9b8e-1e39c8f186df None admin] PATH_INFO: `/v3/domains/default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:01.678271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b06538-75d6-44a8-9b8e-1e39c8f186df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:01.679753 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b06538-75d6-44a8-9b8e-1e39c8f186df None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:01.682241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 96/189] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:01.682863 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b06538-75d6-44a8-9b8e-1e39c8f186df None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:01.706527 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:01.714348 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fdff3-76c2-461e-b3fb-469b7dc38533 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:01.757190 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697d805-a71e-4454-8fd1-e619a76dabd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:01.758025 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697d805-a71e-4454-8fd1-e619a76dabd8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:01.758493 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697d805-a71e-4454-8fd1-e619a76dabd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:01.759075 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697d805-a71e-4454-8fd1-e619a76dabd8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:01.776224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697d805-a71e-4454-8fd1-e619a76dabd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:01.786444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c697d805-a71e-4454-8fd1-e619a76dabd8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:55:01.786444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:01.786444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:01.786444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:01.786444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:01.786444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:01.786444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:01.786444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:01.786444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:01.786444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:01.786444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:01.786444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:01.786444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:01.786444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:01.786444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:01.786444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:01.787379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:01.787379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:01.787379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:01.787379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:01.787379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:01.787379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:01.787379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:01.787379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:01.787379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:01.787379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:01.787379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:01.787379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:01.787379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:01.787379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:01.787379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:01.787379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:01.788267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:01.788267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:01.788267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:01.788267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:01.788267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:01.788267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:01.788267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:01.788267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:01.788267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:01.788267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:01.788267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:55:01.788267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:01.789328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 94/190] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Jul 18 13:55:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:01.790745 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697d805-a71e-4454-8fd1-e619a76dabd8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:01.813469 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b06538-75d6-44a8-9b8e-1e39c8f186df None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:01.850976 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81ff25-7899-404b-a3d2-b2fc88c193f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:01.851645 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81ff25-7899-404b-a3d2-b2fc88c193f3 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:01.851951 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81ff25-7899-404b-a3d2-b2fc88c193f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:01.852250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81ff25-7899-404b-a3d2-b2fc88c193f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:01.853381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81ff25-7899-404b-a3d2-b2fc88c193f3 None admin] RBAC: Adding query filter params (role.id=580aae575f414b3c9b4ccf1e27e0b640, scope.project.id=ec42cd390d9e4d07bb7573020b5b1003, user.id=2ecf077c36454bf782f946577207b833) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:01.853635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81ff25-7899-404b-a3d2-b2fc88c193f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:01.855099 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81ff25-7899-404b-a3d2-b2fc88c193f3 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:01.861199 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 97/191] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:55:01 2022] GET /identity/v3/role_assignments?user.id=2ecf077c36454bf782f946577207b833&scope.project.id=ec42cd390d9e4d07bb7573020b5b1003&role.id=580aae575f414b3c9b4ccf1e27e0b640 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:01.862091 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81ff25-7899-404b-a3d2-b2fc88c193f3 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:01.920443 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697d805-a71e-4454-8fd1-e619a76dabd8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:01.951508 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:01.975983 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562411a-1885-4350-a354-0eacd20d8576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:01.975983 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562411a-1885-4350-a354-0eacd20d8576 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:01.975983 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562411a-1885-4350-a354-0eacd20d8576 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:01.975983 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562411a-1885-4350-a354-0eacd20d8576 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:02.006094 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81ff25-7899-404b-a3d2-b2fc88c193f3 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:02.020502 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:02.064954 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e7713-6578-486c-a48a-0a0764b611d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:02.065666 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e7713-6578-486c-a48a-0a0764b611d0 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:02.065991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e7713-6578-486c-a48a-0a0764b611d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:02.066280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e7713-6578-486c-a48a-0a0764b611d0 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:02.067276 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1e7713-6578-486c-a48a-0a0764b611d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:02.069550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1e7713-6578-486c-a48a-0a0764b611d0 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:02.070783 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562411a-1885-4350-a354-0eacd20d8576 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:02.075122 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562411a-1885-4350-a354-0eacd20d8576 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:02.095783 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 98/192] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Jul 18 13:55:01 2022] POST /identity/v3/users => generated 297 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:55:02.096668 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e7713-6578-486c-a48a-0a0764b611d0 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:02.216824 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 95/193] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:02.218814 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227758eb-5a5f-4d7c-960d-4073aaf29a1e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:02.219417 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227758eb-5a5f-4d7c-960d-4073aaf29a1e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:02.219848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227758eb-5a5f-4d7c-960d-4073aaf29a1e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:02.222295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 96/194] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:02.223282 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227758eb-5a5f-4d7c-960d-4073aaf29a1e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:02.249677 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e7713-6578-486c-a48a-0a0764b611d0 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:02.272588 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:02.294279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2ecc0-d567-40f2-9765-7e1103472c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:02.295208 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2ecc0-d567-40f2-9765-7e1103472c8d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:02.295335 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2ecc0-d567-40f2-9765-7e1103472c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:02.295527 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2ecc0-d567-40f2-9765-7e1103472c8d None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:02.296318 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c2ecc0-d567-40f2-9765-7e1103472c8d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:02.296635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c2ecc0-d567-40f2-9765-7e1103472c8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:02.298030 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c2ecc0-d567-40f2-9765-7e1103472c8d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:02.298845 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:02.308048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 99/195] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:02.309472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495104f-1fa9-4b9a-a1c4-dd36e0e24e5e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:02.309770 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495104f-1fa9-4b9a-a1c4-dd36e0e24e5e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:02.310091 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495104f-1fa9-4b9a-a1c4-dd36e0e24e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:02.327152 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-4495104f-1fa9-4b9a-a1c4-dd36e0e24e5e None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:02.328783 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4495104f-1fa9-4b9a-a1c4-dd36e0e24e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:02.350883 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227758eb-5a5f-4d7c-960d-4073aaf29a1e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:02.394655 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c69400-223d-4a60-9c4d-4c463e30e85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:02.394655 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c69400-223d-4a60-9c4d-4c463e30e85e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:02.394655 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c69400-223d-4a60-9c4d-4c463e30e85e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:02.394655 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c69400-223d-4a60-9c4d-4c463e30e85e None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:02.395658 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c69400-223d-4a60-9c4d-4c463e30e85e None admin] RBAC: Authorizing `identity:get_project(project_id=7bfe28b4f0f648dd906926f955192736)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:02.397013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c69400-223d-4a60-9c4d-4c463e30e85e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:02.398864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 97/196] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:02 2022] GET /identity/v3/projects/7bfe28b4f0f648dd906926f955192736 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:02.400506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c69400-223d-4a60-9c4d-4c463e30e85e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:02.416562 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 100/197] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:02.417753 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495104f-1fa9-4b9a-a1c4-dd36e0e24e5e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:02.518726 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c69400-223d-4a60-9c4d-4c463e30e85e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:02.547332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:02.549447 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495104f-1fa9-4b9a-a1c4-dd36e0e24e5e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:02.557415 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b38a4a-d2c1-4dca-8afc-c07b1953cf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:02.558300 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b38a4a-d2c1-4dca-8afc-c07b1953cf39 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:02.558591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b38a4a-d2c1-4dca-8afc-c07b1953cf39 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:02.558873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b38a4a-d2c1-4dca-8afc-c07b1953cf39 None admin] PATH_INFO: `/v3/users/49eedf7e18eb4505889fa1b78da2b449` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:02.560144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b38a4a-d2c1-4dca-8afc-c07b1953cf39 None admin] RBAC: Authorizing `identity:get_user(user_id=49eedf7e18eb4505889fa1b78da2b449)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:02.561615 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b38a4a-d2c1-4dca-8afc-c07b1953cf39 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:02.563272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 98/198] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:55:02 2022] GET /identity/v3/users/49eedf7e18eb4505889fa1b78da2b449 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:02.563623 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:02.564148 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b38a4a-d2c1-4dca-8afc-c07b1953cf39 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:02.586530 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578af66e-963f-4a31-990e-a5bebbf6febc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:02.587274 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578af66e-963f-4a31-990e-a5bebbf6febc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:02.587568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578af66e-963f-4a31-990e-a5bebbf6febc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:02.587849 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578af66e-963f-4a31-990e-a5bebbf6febc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:02.588785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578af66e-963f-4a31-990e-a5bebbf6febc None admin] RBAC: Adding query filter params (role.id=c60a99de086f45c6820f7872a6b43feb, scope.project.id=7bfe28b4f0f648dd906926f955192736, user.id=49eedf7e18eb4505889fa1b78da2b449) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:02.589150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578af66e-963f-4a31-990e-a5bebbf6febc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:02.590655 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578af66e-963f-4a31-990e-a5bebbf6febc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:02.596969 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 101/199] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:55:02 2022] GET /identity/v3/role_assignments?user.id=49eedf7e18eb4505889fa1b78da2b449&scope.project.id=7bfe28b4f0f648dd906926f955192736&role.id=c60a99de086f45c6820f7872a6b43feb => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:02.597777 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578af66e-963f-4a31-990e-a5bebbf6febc None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:02.755792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b38a4a-d2c1-4dca-8afc-c07b1953cf39 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:02.786942 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578af66e-963f-4a31-990e-a5bebbf6febc None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:02.808535 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c772c72-7701-4138-979e-f53e811c7b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:02.809758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c772c72-7701-4138-979e-f53e811c7b1c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:02.810055 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c772c72-7701-4138-979e-f53e811c7b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:02.810339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c772c72-7701-4138-979e-f53e811c7b1c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:02.826279 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c772c72-7701-4138-979e-f53e811c7b1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:02.829553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2963d2-3983-4890-a3aa-2772efe1b4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:02.830214 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2963d2-3983-4890-a3aa-2772efe1b4eb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:02.830504 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2963d2-3983-4890-a3aa-2772efe1b4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:02.830782 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2963d2-3983-4890-a3aa-2772efe1b4eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:02.835754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-1c772c72-7701-4138-979e-f53e811c7b1c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 13:55:02.835754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:02.835754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:02.835754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:02.835754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:02.835754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:02.835754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:02.835754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:02.835754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:02.835754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:02.835754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:02.835754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:02.835754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:02.835754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:02.835754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:02.835754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:02.836656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:02.836656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:02.836656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:02.836656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:02.836656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:02.836656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:02.836656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:02.836656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:02.836656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:02.836656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:02.836656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:02.836656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:02.836656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:02.836656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:02.836656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:02.836656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:02.837556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:02.837556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:02.837556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:02.837556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:02.837556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:02.837556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:02.837556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:02.837556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:02.837556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:02.837556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:02.837556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 13:55:02.837556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:02.837556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 99/200] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:02.844654 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c772c72-7701-4138-979e-f53e811c7b1c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:02.909802 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:02.948945 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:02.973876 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c772c72-7701-4138-979e-f53e811c7b1c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:03.005999 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2963d2-3983-4890-a3aa-2772efe1b4eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:03.007480 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2963d2-3983-4890-a3aa-2772efe1b4eb None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:03.016642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8233f0a3-b0f0-4190-914d-b98010293676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:03.020502 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233f0a3-b0f0-4190-914d-b98010293676 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:03.021013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233f0a3-b0f0-4190-914d-b98010293676 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:03.021452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233f0a3-b0f0-4190-914d-b98010293676 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:03.022360 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8233f0a3-b0f0-4190-914d-b98010293676 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:03.022819 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8233f0a3-b0f0-4190-914d-b98010293676 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:03.024961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8233f0a3-b0f0-4190-914d-b98010293676 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:03.040847 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 100/201] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:55:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:03.047927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8233f0a3-b0f0-4190-914d-b98010293676 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:03.215190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 102/202] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:03.220664 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2963d2-3983-4890-a3aa-2772efe1b4eb None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:03.246835 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8233f0a3-b0f0-4190-914d-b98010293676 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:03.269541 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:03.289638 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24815692-a9a7-4345-9f32-8796af6c3ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:03.291312 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24815692-a9a7-4345-9f32-8796af6c3ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:03.291749 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24815692-a9a7-4345-9f32-8796af6c3ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:03.292212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24815692-a9a7-4345-9f32-8796af6c3ad9 None admin] PATH_INFO: `/v3/users/f2f07f4728f74a2fb27ea83e4cc23351` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:03.294471 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24815692-a9a7-4345-9f32-8796af6c3ad9 None admin] RBAC: Authorizing `identity:get_user(user_id=f2f07f4728f74a2fb27ea83e4cc23351)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:03.296108 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24815692-a9a7-4345-9f32-8796af6c3ad9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:03.297900 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 101/203] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:55:03 2022] GET /identity/v3/users/f2f07f4728f74a2fb27ea83e4cc23351 => generated 289 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:03.303921 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24815692-a9a7-4345-9f32-8796af6c3ad9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:03.416343 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24815692-a9a7-4345-9f32-8796af6c3ad9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:03.430032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:03.459734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef013e-9bde-493e-9043-18886420d23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:03.460426 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef013e-9bde-493e-9043-18886420d23b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:03.460717 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef013e-9bde-493e-9043-18886420d23b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:03.460999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef013e-9bde-493e-9043-18886420d23b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:03.481373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2963d2-3983-4890-a3aa-2772efe1b4eb None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:03.527025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe772d3-d936-46e6-8ac7-0c3be5595f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:03.527905 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe772d3-d936-46e6-8ac7-0c3be5595f0d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:03.528216 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe772d3-d936-46e6-8ac7-0c3be5595f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:03.528494 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe772d3-d936-46e6-8ac7-0c3be5595f0d None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:03.532833 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe772d3-d936-46e6-8ac7-0c3be5595f0d None admin] RBAC: Authorizing `identity:get_project(project_id=7bfe28b4f0f648dd906926f955192736)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:03.540942 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe772d3-d936-46e6-8ac7-0c3be5595f0d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:03.551585 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 103/204] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:03 2022] GET /identity/v3/projects/7bfe28b4f0f648dd906926f955192736 => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:03.552559 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe772d3-d936-46e6-8ac7-0c3be5595f0d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:03.565271 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:03.591317 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef013e-9bde-493e-9043-18886420d23b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:03.592827 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef013e-9bde-493e-9043-18886420d23b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:03.747025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe772d3-d936-46e6-8ac7-0c3be5595f0d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:03.785250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dde945-af76-47f7-b693-7497e33751e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:03.785911 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dde945-af76-47f7-b693-7497e33751e9 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:03.786198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dde945-af76-47f7-b693-7497e33751e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:03.786479 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dde945-af76-47f7-b693-7497e33751e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:03.787302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dde945-af76-47f7-b693-7497e33751e9 None admin] RBAC: Adding query filter params (role.id=580aae575f414b3c9b4ccf1e27e0b640, scope.project.id=7bfe28b4f0f648dd906926f955192736, user.id=49eedf7e18eb4505889fa1b78da2b449) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:03.787616 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dde945-af76-47f7-b693-7497e33751e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:03.789085 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dde945-af76-47f7-b693-7497e33751e9 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:03.794361 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 102/205] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:03.799703 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef013e-9bde-493e-9043-18886420d23b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:03.807992 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 104/206] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:55:03 2022] GET /identity/v3/role_assignments?user.id=49eedf7e18eb4505889fa1b78da2b449&scope.project.id=7bfe28b4f0f648dd906926f955192736&role.id=580aae575f414b3c9b4ccf1e27e0b640 => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:03.921444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:03.940334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef013e-9bde-493e-9043-18886420d23b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:03.991448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f6eb3-e6ca-41e6-9b9b-7ff7a421f844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:03.992361 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f6eb3-e6ca-41e6-9b9b-7ff7a421f844 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:03.992641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f6eb3-e6ca-41e6-9b9b-7ff7a421f844 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:03.993835 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f6eb3-e6ca-41e6-9b9b-7ff7a421f844 None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:03.996751 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f6eb3-e6ca-41e6-9b9b-7ff7a421f844 None admin] RBAC: Authorizing `identity:get_project(project_id=ec42cd390d9e4d07bb7573020b5b1003)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:03.998252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f6eb3-e6ca-41e6-9b9b-7ff7a421f844 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:04.001036 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 103/207] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:03 2022] GET /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:04.009701 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dde945-af76-47f7-b693-7497e33751e9 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:04.027076 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:04.146212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dde945-af76-47f7-b693-7497e33751e9 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:04.187402 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568753ca-ef22-40b3-85da-05579e5302ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:04.188097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568753ca-ef22-40b3-85da-05579e5302ae None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:04.188391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568753ca-ef22-40b3-85da-05579e5302ae None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:04.188669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568753ca-ef22-40b3-85da-05579e5302ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:04.189507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568753ca-ef22-40b3-85da-05579e5302ae None admin] RBAC: Adding query filter params (role.id=fd276ad89f2d4192b4f90e126d9362d3, scope.project.id=ec42cd390d9e4d07bb7573020b5b1003, user.id=f2f07f4728f74a2fb27ea83e4cc23351) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:04.189817 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568753ca-ef22-40b3-85da-05579e5302ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:04.191268 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568753ca-ef22-40b3-85da-05579e5302ae None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:04.197980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 105/208] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:55:04 2022] GET /identity/v3/role_assignments?user.id=f2f07f4728f74a2fb27ea83e4cc23351&scope.project.id=ec42cd390d9e4d07bb7573020b5b1003&role.id=fd276ad89f2d4192b4f90e126d9362d3 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:04.425150 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e898b3-e08c-4fd6-90d1-2e1a42b2a15c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:04.425483 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e898b3-e08c-4fd6-90d1-2e1a42b2a15c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:04.425671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e898b3-e08c-4fd6-90d1-2e1a42b2a15c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:04.427305 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 104/209] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:04.432777 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e13460-84ab-476e-bf19-bda3a7064d13 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:04.433071 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e13460-84ab-476e-bf19-bda3a7064d13 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:04.433354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e13460-84ab-476e-bf19-bda3a7064d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:04.451250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-95e13460-84ab-476e-bf19-bda3a7064d13 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:04.452483 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-95e13460-84ab-476e-bf19-bda3a7064d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:04.529904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 106/210] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:04.554357 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e898b3-e08c-4fd6-90d1-2e1a42b2a15c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:04.684446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e898b3-e08c-4fd6-90d1-2e1a42b2a15c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:04.725258 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3601a055-42aa-41d7-9b96-7fd420c9395d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:04.726071 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601a055-42aa-41d7-9b96-7fd420c9395d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:04.726364 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601a055-42aa-41d7-9b96-7fd420c9395d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:04.726649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601a055-42aa-41d7-9b96-7fd420c9395d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:04.727890 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3601a055-42aa-41d7-9b96-7fd420c9395d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:04.729410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3601a055-42aa-41d7-9b96-7fd420c9395d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:04.731373 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 105/211] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:04.736587 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e13460-84ab-476e-bf19-bda3a7064d13 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:04.738769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:04.841672 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b69e0-5948-4550-a746-04143409991d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:04.841996 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b69e0-5948-4550-a746-04143409991d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:04.847603 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b69e0-5948-4550-a746-04143409991d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:04.847759 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e13460-84ab-476e-bf19-bda3a7064d13 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:04.847842 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 106/212] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:55:04.849402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e760772-6eb5-4fd6-893c-80bb7ee49aad None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:04.849899 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e760772-6eb5-4fd6-893c-80bb7ee49aad None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:04.850400 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e760772-6eb5-4fd6-893c-80bb7ee49aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:04.867756 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1e760772-6eb5-4fd6-893c-80bb7ee49aad None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:04.869315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1e760772-6eb5-4fd6-893c-80bb7ee49aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:04.882569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8a7fe-aa83-44d4-b2df-c758940fb64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:04.883249 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8a7fe-aa83-44d4-b2df-c758940fb64d None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:04.884085 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8a7fe-aa83-44d4-b2df-c758940fb64d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:04.884369 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8a7fe-aa83-44d4-b2df-c758940fb64d None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:04.885074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d8a7fe-aa83-44d4-b2df-c758940fb64d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:04.886496 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:04.886702 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d8a7fe-aa83-44d4-b2df-c758940fb64d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:04.910991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 107/213] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Jul 18 13:55:04 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 13:55:04.962278 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 107/214] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:04.986920 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c428ae0-6f60-48ed-9d84-429d531badcc None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:04.987226 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c428ae0-6f60-48ed-9d84-429d531badcc None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:04.987458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e760772-6eb5-4fd6-893c-80bb7ee49aad None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:04.987584 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c428ae0-6f60-48ed-9d84-429d531badcc None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:04.988941 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 108/215] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:04.994829 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e39125-644f-4d50-ba88-7cab24a9b718 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:04.995121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e39125-644f-4d50-ba88-7cab24a9b718 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:04.995404 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e39125-644f-4d50-ba88-7cab24a9b718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:05.013778 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b3e39125-644f-4d50-ba88-7cab24a9b718 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:05.015002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b3e39125-644f-4d50-ba88-7cab24a9b718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:05.085048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:05.110748 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 109/216] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 13:55:05.121775 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e760772-6eb5-4fd6-893c-80bb7ee49aad None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:05.132559 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e39125-644f-4d50-ba88-7cab24a9b718 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:05.160122 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c77d57-9ea9-46ac-b5a7-e2ce910e5e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:05.162380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c77d57-9ea9-46ac-b5a7-e2ce910e5e99 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:05.162994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c77d57-9ea9-46ac-b5a7-e2ce910e5e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:05.164642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c77d57-9ea9-46ac-b5a7-e2ce910e5e99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:05.183723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c77d57-9ea9-46ac-b5a7-e2ce910e5e99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:05.193490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-d4c77d57-9ea9-46ac-b5a7-e2ce910e5e99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:05.193490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:05.193490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:05.193490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:05.193490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:05.193490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:05.193490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:05.193490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:05.193490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:05.193490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:05.193490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:05.193490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:05.193490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:05.193490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:05.193490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:05.193490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:05.194594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:05.194594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:05.194594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:05.194594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:05.194594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:05.194594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:05.194594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:05.194594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:05.194594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:05.194594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:05.194594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:05.194594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:05.194594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:05.194594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:05.194594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:05.194594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:05.195817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:05.195817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:05.195817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:05.195817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:05.195817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:05.195817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:05.195817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:05.195817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:05.195817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:05.195817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:05.195817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:05.195817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:05.195817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 108/217] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:55:05.201229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c77d57-9ea9-46ac-b5a7-e2ce910e5e99 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:05.217584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:05.256789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e39125-644f-4d50-ba88-7cab24a9b718 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:05.301835 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c4370-63fd-4499-9550-9946414b876a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:05.302695 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c4370-63fd-4499-9550-9946414b876a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:05.302982 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c4370-63fd-4499-9550-9946414b876a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:05.303260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c4370-63fd-4499-9550-9946414b876a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:05.317222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c4370-63fd-4499-9550-9946414b876a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:05.326569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-e87c4370-63fd-4499-9550-9946414b876a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:55:05.326569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:05.326569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:05.326569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:05.326569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:05.326569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:05.326569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:05.326569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:05.326569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:05.326569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:05.326569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:05.326569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:05.326569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:05.326569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:05.326569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:05.326569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:05.327513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:05.327513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:05.327513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:05.327513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:05.327513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:05.327513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:05.327513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:05.327513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:05.327513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:05.327513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:05.327513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:05.327513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:05.327513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:05.327513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:05.327513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:05.327513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:05.328391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:05.328391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:05.328391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:05.328391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:05.328391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:05.328391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:05.328391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:05.328391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:05.328391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:05.328391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:05.328391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:55:05.328391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:05.330976 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 110/218] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:55:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:55:05.335620 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c4370-63fd-4499-9550-9946414b876a None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:05.342544 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c77d57-9ea9-46ac-b5a7-e2ce910e5e99 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:05.383851 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e137559-349a-41a7-8597-e6fa82ed46c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:05.384613 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e137559-349a-41a7-8597-e6fa82ed46c6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:05.385345 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e137559-349a-41a7-8597-e6fa82ed46c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:05.385647 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e137559-349a-41a7-8597-e6fa82ed46c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:05.386408 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e137559-349a-41a7-8597-e6fa82ed46c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:05.386717 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e137559-349a-41a7-8597-e6fa82ed46c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:05.388895 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e137559-349a-41a7-8597-e6fa82ed46c6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:05.396301 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:05.400172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 109/219] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:55:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:05.406728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e137559-349a-41a7-8597-e6fa82ed46c6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:05.456913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c4370-63fd-4499-9550-9946414b876a None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:05.484168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:05.492581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f2bf0-1376-4836-90bb-c890c0fce165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:05.493800 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f2bf0-1376-4836-90bb-c890c0fce165 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:05.494087 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f2bf0-1376-4836-90bb-c890c0fce165 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:05.494365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f2bf0-1376-4836-90bb-c890c0fce165 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:05.495124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f2bf0-1376-4836-90bb-c890c0fce165 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:05.495436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f2bf0-1376-4836-90bb-c890c0fce165 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:05.496853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f2bf0-1376-4836-90bb-c890c0fce165 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:05.506803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 111/220] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:05.513471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f2bf0-1376-4836-90bb-c890c0fce165 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:05.534386 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e137559-349a-41a7-8597-e6fa82ed46c6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:05.573239 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594031ee-974c-4b20-afc5-7dd75f58e273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:05.574495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594031ee-974c-4b20-afc5-7dd75f58e273 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:05.574914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594031ee-974c-4b20-afc5-7dd75f58e273 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:05.575206 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594031ee-974c-4b20-afc5-7dd75f58e273 None admin] PATH_INFO: `/v3/users/7867709f132b48fe9721420e8f43a191` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:05.607172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594031ee-974c-4b20-afc5-7dd75f58e273 None admin] RBAC: Authorizing `identity:get_user(user_id=7867709f132b48fe9721420e8f43a191)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:05.608711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594031ee-974c-4b20-afc5-7dd75f58e273 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:05.610472 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 110/221] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:55:05 2022] GET /identity/v3/users/7867709f132b48fe9721420e8f43a191 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:05.633467 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594031ee-974c-4b20-afc5-7dd75f58e273 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:05.702531 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f2bf0-1376-4836-90bb-c890c0fce165 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:05.726153 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:05.773571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee192a31-110d-460b-85b1-c188ca72a228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:05.775033 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee192a31-110d-460b-85b1-c188ca72a228 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:05.775330 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee192a31-110d-460b-85b1-c188ca72a228 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:05.775613 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee192a31-110d-460b-85b1-c188ca72a228 None admin] PATH_INFO: `/v3/users/49eedf7e18eb4505889fa1b78da2b449` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:05.777989 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee192a31-110d-460b-85b1-c188ca72a228 None admin] RBAC: Authorizing `identity:get_user(user_id=49eedf7e18eb4505889fa1b78da2b449)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:05.779466 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee192a31-110d-460b-85b1-c188ca72a228 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:05.781349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 112/222] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:55:05 2022] GET /identity/v3/users/49eedf7e18eb4505889fa1b78da2b449 => generated 283 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:05.787873 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee192a31-110d-460b-85b1-c188ca72a228 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:05.816698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594031ee-974c-4b20-afc5-7dd75f58e273 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:05.882070 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:05.894462 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92524ab2-5ca8-4665-9389-0536e63459fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:05.895130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92524ab2-5ca8-4665-9389-0536e63459fc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:05.895421 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92524ab2-5ca8-4665-9389-0536e63459fc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:05.895703 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92524ab2-5ca8-4665-9389-0536e63459fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:05.972711 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee192a31-110d-460b-85b1-c188ca72a228 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:05.984954 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92524ab2-5ca8-4665-9389-0536e63459fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:05.986932 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92524ab2-5ca8-4665-9389-0536e63459fc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:06.044846 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c1976-afb9-4270-af82-ab416bccdefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:06.046001 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c1976-afb9-4270-af82-ab416bccdefb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:06.046291 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c1976-afb9-4270-af82-ab416bccdefb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:06.046605 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c1976-afb9-4270-af82-ab416bccdefb None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:06.048457 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c1976-afb9-4270-af82-ab416bccdefb None admin] RBAC: Authorizing `identity:get_project(project_id=7bfe28b4f0f648dd906926f955192736)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:06.050602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c1976-afb9-4270-af82-ab416bccdefb None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:06.052627 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 113/223] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:05 2022] GET /identity/v3/projects/7bfe28b4f0f648dd906926f955192736 => generated 302 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:06.054375 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c588566-683e-4437-8018-9841d896b11e None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:06.054796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c588566-683e-4437-8018-9841d896b11e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:06.055214 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c588566-683e-4437-8018-9841d896b11e None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:06.060020 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 114/224] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:06.060817 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c588566-683e-4437-8018-9841d896b11e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:06.100273 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 111/225] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 13:55:06.106121 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbfd42-3547-46d5-a193-6e02ca340c6d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:06.106424 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbfd42-3547-46d5-a193-6e02ca340c6d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:06.106798 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fbfd42-3547-46d5-a193-6e02ca340c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:06.123035 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-21fbfd42-3547-46d5-a193-6e02ca340c6d None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:06.124374 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-21fbfd42-3547-46d5-a193-6e02ca340c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:06.124490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:06.152336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:06.199973 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 112/226] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:06.199973 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fbfd42-3547-46d5-a193-6e02ca340c6d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:06.207965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c588566-683e-4437-8018-9841d896b11e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:06.253557 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d878441-8b7e-4045-8731-02627384cbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:06.254381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d878441-8b7e-4045-8731-02627384cbba None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:06.254670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d878441-8b7e-4045-8731-02627384cbba None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:06.254951 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d878441-8b7e-4045-8731-02627384cbba None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736/users/49eedf7e18eb4505889fa1b78da2b449/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:06.256993 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d878441-8b7e-4045-8731-02627384cbba None admin] RBAC: Authorizing `identity:create_grant(project_id=7bfe28b4f0f648dd906926f955192736, user_id=49eedf7e18eb4505889fa1b78da2b449, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:06.258515 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d878441-8b7e-4045-8731-02627384cbba None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:06.275564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 115/227] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:55:06 2022] PUT /identity/v3/projects/7bfe28b4f0f648dd906926f955192736/users/49eedf7e18eb4505889fa1b78da2b449/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:55:06.276514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d878441-8b7e-4045-8731-02627384cbba None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:06.340001 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fbfd42-3547-46d5-a193-6e02ca340c6d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:06.422080 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba962f-0438-4c04-a7b2-781c19c9cad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:06.423336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba962f-0438-4c04-a7b2-781c19c9cad1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:06.423824 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba962f-0438-4c04-a7b2-781c19c9cad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:06.424331 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba962f-0438-4c04-a7b2-781c19c9cad1 None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:06.424941 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d878441-8b7e-4045-8731-02627384cbba None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:06.426068 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba962f-0438-4c04-a7b2-781c19c9cad1 None admin] RBAC: Authorizing `identity:get_project(project_id=7bfe28b4f0f648dd906926f955192736)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:06.428015 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba962f-0438-4c04-a7b2-781c19c9cad1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:06.435980 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 113/228] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:06 2022] GET /identity/v3/projects/7bfe28b4f0f648dd906926f955192736 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:06.437853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:06.441116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba962f-0438-4c04-a7b2-781c19c9cad1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:06.478684 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a8a6d-b5b9-4833-9419-9f9b4cdfe52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:06.479515 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a8a6d-b5b9-4833-9419-9f9b4cdfe52b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:06.479803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a8a6d-b5b9-4833-9419-9f9b4cdfe52b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:06.480132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a8a6d-b5b9-4833-9419-9f9b4cdfe52b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:06.491921 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a8a6d-b5b9-4833-9419-9f9b4cdfe52b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:06.492553 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:06.501823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-b20a8a6d-b5b9-4833-9419-9f9b4cdfe52b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:55:06.501823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:06.501823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:06.501823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:06.501823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:06.501823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:06.501823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:06.501823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:06.501823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:06.501823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:06.501823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:06.501823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:06.501823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:06.501823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:06.501823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:06.501823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:06.502718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:06.502718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:06.502718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:06.502718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:06.502718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:06.502718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:06.502718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:06.502718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:06.502718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:06.502718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:06.502718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:06.502718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:06.502718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:06.502718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:06.502718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:06.502718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:06.503512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:06.503512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:06.503512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:06.503512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:06.503512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:06.503512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:06.503512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:06.503512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:06.503512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:06.503512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:06.503512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:55:06.503512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:06.503512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 116/229] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Jul 18 13:55:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:06.516864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a8a6d-b5b9-4833-9419-9f9b4cdfe52b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:06.569463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba962f-0438-4c04-a7b2-781c19c9cad1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:06.626266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59ff7f-0895-4aa7-b64f-be594a551e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:06.626944 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59ff7f-0895-4aa7-b64f-be594a551e7c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:06.627231 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59ff7f-0895-4aa7-b64f-be594a551e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:06.627507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59ff7f-0895-4aa7-b64f-be594a551e7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:06.628376 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59ff7f-0895-4aa7-b64f-be594a551e7c None admin] RBAC: Adding query filter params (role.id=1399a838106b4f9bb5fa1c942cc3ef22, scope.project.id=7bfe28b4f0f648dd906926f955192736, user.id=7867709f132b48fe9721420e8f43a191) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:06.628692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59ff7f-0895-4aa7-b64f-be594a551e7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:06.630150 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e59ff7f-0895-4aa7-b64f-be594a551e7c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:06.648807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 114/230] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:55:06 2022] GET /identity/v3/role_assignments?user.id=7867709f132b48fe9721420e8f43a191&scope.project.id=7bfe28b4f0f648dd906926f955192736&role.id=1399a838106b4f9bb5fa1c942cc3ef22 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:06.679485 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a8a6d-b5b9-4833-9419-9f9b4cdfe52b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:06.717543 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9b5f0-b5da-4e34-81b9-40461870c174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:06.718216 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9b5f0-b5da-4e34-81b9-40461870c174 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:06.718502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9b5f0-b5da-4e34-81b9-40461870c174 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:06.718777 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9b5f0-b5da-4e34-81b9-40461870c174 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:06.719523 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9b5f0-b5da-4e34-81b9-40461870c174 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:06.719826 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9b5f0-b5da-4e34-81b9-40461870c174 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:06.721256 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9b5f0-b5da-4e34-81b9-40461870c174 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:06.730115 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 117/231] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:06.737123 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59ff7f-0895-4aa7-b64f-be594a551e7c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:06.881717 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:06.897183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59ff7f-0895-4aa7-b64f-be594a551e7c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:06.935083 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05495e8b-710e-4e60-a638-47d09c4c6fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:06.935985 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05495e8b-710e-4e60-a638-47d09c4c6fff None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:06.936280 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05495e8b-710e-4e60-a638-47d09c4c6fff None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:06.936563 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05495e8b-710e-4e60-a638-47d09c4c6fff None admin] PATH_INFO: `/v3/users/49eedf7e18eb4505889fa1b78da2b449` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:06.939567 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05495e8b-710e-4e60-a638-47d09c4c6fff None admin] RBAC: Authorizing `identity:get_user(user_id=49eedf7e18eb4505889fa1b78da2b449)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:06.939567 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05495e8b-710e-4e60-a638-47d09c4c6fff None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:06.941343 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 115/232] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:55:06 2022] GET /identity/v3/users/49eedf7e18eb4505889fa1b78da2b449 => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:06.948928 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9b5f0-b5da-4e34-81b9-40461870c174 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:07.007135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:07.110803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9b5f0-b5da-4e34-81b9-40461870c174 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:07.145977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fde6db-efa9-4567-a144-4deb853aa629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:07.146770 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fde6db-efa9-4567-a144-4deb853aa629 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:07.147061 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fde6db-efa9-4567-a144-4deb853aa629 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:07.147345 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fde6db-efa9-4567-a144-4deb853aa629 None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:07.148584 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fde6db-efa9-4567-a144-4deb853aa629 None admin] RBAC: Authorizing `identity:get_project(project_id=7bfe28b4f0f648dd906926f955192736)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:07.150106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fde6db-efa9-4567-a144-4deb853aa629 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:07.151823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 118/233] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:06 2022] GET /identity/v3/projects/7bfe28b4f0f648dd906926f955192736 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:07.158473 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05495e8b-710e-4e60-a638-47d09c4c6fff None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:07.283387 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05495e8b-710e-4e60-a638-47d09c4c6fff None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:07.331536 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b14bf1-489d-4c83-a489-12698e20e333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:07.332836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b14bf1-489d-4c83-a489-12698e20e333 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:07.333323 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b14bf1-489d-4c83-a489-12698e20e333 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:07.333796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b14bf1-489d-4c83-a489-12698e20e333 None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736/users/49eedf7e18eb4505889fa1b78da2b449/roles/580aae575f414b3c9b4ccf1e27e0b640` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:07.336301 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b14bf1-489d-4c83-a489-12698e20e333 None admin] RBAC: Authorizing `identity:create_grant(project_id=7bfe28b4f0f648dd906926f955192736, user_id=49eedf7e18eb4505889fa1b78da2b449, role_id=580aae575f414b3c9b4ccf1e27e0b640)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:07.338151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b14bf1-489d-4c83-a489-12698e20e333 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:07.368146 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 116/234] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:55:07 2022] PUT /identity/v3/projects/7bfe28b4f0f648dd906926f955192736/users/49eedf7e18eb4505889fa1b78da2b449/roles/580aae575f414b3c9b4ccf1e27e0b640 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:55:07.616219 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c2d87-4a3c-4e66-aa9e-39d67d2d435f None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:07.616430 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c2d87-4a3c-4e66-aa9e-39d67d2d435f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:07.616981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c2d87-4a3c-4e66-aa9e-39d67d2d435f None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:07.619173 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 119/235] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:55:07.625342 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1d9f9-cfcb-4423-a8d9-1e1d8304a511 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:07.625939 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1d9f9-cfcb-4423-a8d9-1e1d8304a511 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:07.626427 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1d9f9-cfcb-4423-a8d9-1e1d8304a511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:07.648964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-83f1d9f9-cfcb-4423-a8d9-1e1d8304a511 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:07.651141 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-83f1d9f9-cfcb-4423-a8d9-1e1d8304a511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:07.666344 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:07.724792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 117/236] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:07.749551 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c2d87-4a3c-4e66-aa9e-39d67d2d435f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:07.757998 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c5365-3270-475a-9f0e-fbc8acd519a5 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:07.758656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c5365-3270-475a-9f0e-fbc8acd519a5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:07.759174 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c5365-3270-475a-9f0e-fbc8acd519a5 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:07.760865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 118/237] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:07.766626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95742b5a-d183-4081-acd2-31116e87a249 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:07.766759 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95742b5a-d183-4081-acd2-31116e87a249 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:07.767010 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95742b5a-d183-4081-acd2-31116e87a249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:07.785666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-95742b5a-d183-4081-acd2-31116e87a249 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:07.787130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-95742b5a-d183-4081-acd2-31116e87a249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:07.899725 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:07.918235 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c2d87-4a3c-4e66-aa9e-39d67d2d435f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:07.920666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 119/238] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 13:55:07.975706 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95742b5a-d183-4081-acd2-31116e87a249 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:07.988510 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e9117-9447-4c8f-86f1-e8c538c4624b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:07.989354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e9117-9447-4c8f-86f1-e8c538c4624b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:07.989649 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e9117-9447-4c8f-86f1-e8c538c4624b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:07.989934 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e9117-9447-4c8f-86f1-e8c538c4624b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:07.994370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e9117-9447-4c8f-86f1-e8c538c4624b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:07.995893 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e9117-9447-4c8f-86f1-e8c538c4624b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:08.008115 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 120/239] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:08.019668 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e9117-9447-4c8f-86f1-e8c538c4624b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:08.137688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:08.177988 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e9117-9447-4c8f-86f1-e8c538c4624b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:08.178471 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95742b5a-d183-4081-acd2-31116e87a249 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:08.215092 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf5f7d-7420-413e-9446-f76e65ad6263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:08.216204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf5f7d-7420-413e-9446-f76e65ad6263 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:08.216722 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf5f7d-7420-413e-9446-f76e65ad6263 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:08.217201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf5f7d-7420-413e-9446-f76e65ad6263 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:08.218206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf5f7d-7420-413e-9446-f76e65ad6263 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:08.219883 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf5f7d-7420-413e-9446-f76e65ad6263 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:08.247643 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee007afb-5fb5-471f-9b5e-5b4dee0e424f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:08.248890 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee007afb-5fb5-471f-9b5e-5b4dee0e424f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:08.250097 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee007afb-5fb5-471f-9b5e-5b4dee0e424f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:08.250586 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 121/240] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Jul 18 13:55:08 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:55:08.251118 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee007afb-5fb5-471f-9b5e-5b4dee0e424f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:08.264986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee007afb-5fb5-471f-9b5e-5b4dee0e424f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:08.273446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ee007afb-5fb5-471f-9b5e-5b4dee0e424f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 13:55:08.273446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:08.273446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:08.273446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:08.273446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:08.273446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:08.273446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:08.273446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:08.273446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:08.273446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:08.273446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:08.273446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:08.273446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:08.273446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:08.273446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:08.273446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:08.274850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:08.274850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:08.274850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:08.274850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:08.274850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:08.274850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:08.274850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:08.274850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:08.274850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:08.274850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:08.274850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:08.274850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:08.274850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:08.274850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:08.274850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:08.274850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:08.275649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:08.275649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:08.275649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:08.275649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:08.275649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:08.275649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:08.275649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:08.275649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:08.275649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:08.275649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:08.275649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 13:55:08.275649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:08.278064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 120/241] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:55:08.284152 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf5f7d-7420-413e-9446-f76e65ad6263 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:08.323686 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:08.433143 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf5f7d-7420-413e-9446-f76e65ad6263 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:08.489715 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aefddf-b51d-43bc-83d9-3df83a3508fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:08.490423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aefddf-b51d-43bc-83d9-3df83a3508fc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:08.490719 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aefddf-b51d-43bc-83d9-3df83a3508fc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:08.491003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aefddf-b51d-43bc-83d9-3df83a3508fc None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:08.491777 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aefddf-b51d-43bc-83d9-3df83a3508fc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:08.492117 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aefddf-b51d-43bc-83d9-3df83a3508fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:08.494094 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aefddf-b51d-43bc-83d9-3df83a3508fc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:08.508204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 122/242] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:55:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:08.513889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee007afb-5fb5-471f-9b5e-5b4dee0e424f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:08.644274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee007afb-5fb5-471f-9b5e-5b4dee0e424f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:08.678051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0260e1-074c-413a-86f0-647b5e29ed18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:08.678918 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0260e1-074c-413a-86f0-647b5e29ed18 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:08.679207 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0260e1-074c-413a-86f0-647b5e29ed18 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:08.679485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0260e1-074c-413a-86f0-647b5e29ed18 None admin] PATH_INFO: `/v3/users/f30246b5afe84bdeaee3960a3bd3117c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:08.689972 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:08.697688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0260e1-074c-413a-86f0-647b5e29ed18 None admin] RBAC: Authorizing `identity:get_user(user_id=f30246b5afe84bdeaee3960a3bd3117c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:08.699277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0260e1-074c-413a-86f0-647b5e29ed18 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:08.701478 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 121/243] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:55:08 2022] GET /identity/v3/users/f30246b5afe84bdeaee3960a3bd3117c => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:08.707374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aefddf-b51d-43bc-83d9-3df83a3508fc None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:08.809837 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:08.832756 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aefddf-b51d-43bc-83d9-3df83a3508fc None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:08.853215 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44be774-41cc-4dea-9874-0b92c57ddbcf None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:08.853847 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44be774-41cc-4dea-9874-0b92c57ddbcf None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:08.854346 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44be774-41cc-4dea-9874-0b92c57ddbcf None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:08.855967 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 122/244] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:08.861437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d801a-4281-47dd-8546-d3c4e23fda91 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:08.862031 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d801a-4281-47dd-8546-d3c4e23fda91 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:08.862510 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d801a-4281-47dd-8546-d3c4e23fda91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:08.874783 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c33e8c-ead3-4b1b-a3d7-65d40dd651f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:08.875492 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c33e8c-ead3-4b1b-a3d7-65d40dd651f6 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:08.875786 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c33e8c-ead3-4b1b-a3d7-65d40dd651f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:08.878467 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c33e8c-ead3-4b1b-a3d7-65d40dd651f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:08.881949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1f7d801a-4281-47dd-8546-d3c4e23fda91 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:08.883459 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1f7d801a-4281-47dd-8546-d3c4e23fda91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:08.969062 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 123/245] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 13:55:08.981168 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c33e8c-ead3-4b1b-a3d7-65d40dd651f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:08.982775 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c33e8c-ead3-4b1b-a3d7-65d40dd651f6 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:08.988844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cf8de-b455-46dc-b97e-830439c8578d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:08.989400 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cf8de-b455-46dc-b97e-830439c8578d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:08.989882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cf8de-b455-46dc-b97e-830439c8578d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:08.991696 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 124/246] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:55:08.996888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0163f-69b5-4d19-a40c-acd66e3ec5d4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:08.997356 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0163f-69b5-4d19-a40c-acd66e3ec5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:08.997782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0163f-69b5-4d19-a40c-acd66e3ec5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:09.016241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8de0163f-69b5-4d19-a40c-acd66e3ec5d4 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:09.017452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8de0163f-69b5-4d19-a40c-acd66e3ec5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:09.077037 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:09.092358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 125/247] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 13:55:09.093746 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0163f-69b5-4d19-a40c-acd66e3ec5d4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:09.099572 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 123/248] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:09.105285 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c33e8c-ead3-4b1b-a3d7-65d40dd651f6 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:09.174460 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:09.215235 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0163f-69b5-4d19-a40c-acd66e3ec5d4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:09.239901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c33e8c-ead3-4b1b-a3d7-65d40dd651f6 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:09.253225 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455cb8df-c565-4373-a943-00767324e3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:09.254031 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cb8df-c565-4373-a943-00767324e3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:09.254332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cb8df-c565-4373-a943-00767324e3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:09.254586 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cb8df-c565-4373-a943-00767324e3b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:09.265899 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455cb8df-c565-4373-a943-00767324e3b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:09.273572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-455cb8df-c565-4373-a943-00767324e3b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:55:09.273572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:09.273572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:09.273572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:09.273572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:09.273572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:09.273572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:09.273572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:09.273572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:09.273572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:09.273572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:09.273572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:09.273572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:09.273572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:09.273572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:09.273572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:09.275078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:09.275078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:09.275078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:09.275078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:09.275078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:09.275078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:09.275078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:09.275078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:09.275078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:09.275078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:09.275078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:09.275078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:09.275078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:09.275078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:09.275078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:09.275078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:09.275865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:09.275865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:09.275865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:09.275865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:09.275865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:09.275865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:09.275865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:09.275865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:09.275865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:09.275865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:09.275865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:55:09.275865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:09.275865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 126/249] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:55:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:09.276554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455cb8df-c565-4373-a943-00767324e3b8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:09.291339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d5015-8526-4488-81e2-5da7bfd0f2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:09.292872 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d5015-8526-4488-81e2-5da7bfd0f2c0 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:09.293361 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d5015-8526-4488-81e2-5da7bfd0f2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:09.294095 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d5015-8526-4488-81e2-5da7bfd0f2c0 None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:09.296360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d5015-8526-4488-81e2-5da7bfd0f2c0 None admin] RBAC: Authorizing `identity:get_project(project_id=7bfe28b4f0f648dd906926f955192736)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:09.298672 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d5015-8526-4488-81e2-5da7bfd0f2c0 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:09.300640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 124/250] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:09 2022] GET /identity/v3/projects/7bfe28b4f0f648dd906926f955192736 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:09.302304 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d5015-8526-4488-81e2-5da7bfd0f2c0 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:09.363115 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:09.383247 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455cb8df-c565-4373-a943-00767324e3b8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:09.401763 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:09.420190 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a6566-5b3e-421b-91a9-a9f967f2435f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:09.421083 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a6566-5b3e-421b-91a9-a9f967f2435f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:09.421444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a6566-5b3e-421b-91a9-a9f967f2435f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:09.421791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a6566-5b3e-421b-91a9-a9f967f2435f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:09.422627 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d5015-8526-4488-81e2-5da7bfd0f2c0 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:09.433021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a6566-5b3e-421b-91a9-a9f967f2435f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:09.447830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-700a6566-5b3e-421b-91a9-a9f967f2435f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:09.447830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:09.447830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:09.447830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:09.447830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:09.447830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:09.447830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:09.447830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:09.447830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:09.447830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:09.447830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:09.447830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:09.447830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:09.447830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:09.447830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:09.447830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:09.448795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:09.448795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:09.448795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:09.448795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:09.448795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:09.448795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:09.448795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:09.448795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:09.448795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:09.448795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:09.448795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:09.448795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:09.448795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:09.448795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:09.448795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:09.448795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:09.449579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:09.449579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:09.449579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:09.449579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:09.449579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:09.449579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:09.449579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:09.449579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:09.449579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:09.449579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:09.449579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:09.449579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:09.449579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 127/251] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:09.450228 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a6566-5b3e-421b-91a9-a9f967f2435f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:09.468222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93699b5e-8f56-485f-b9ec-201de6f42f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:09.469184 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93699b5e-8f56-485f-b9ec-201de6f42f88 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:09.469670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93699b5e-8f56-485f-b9ec-201de6f42f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:09.470224 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93699b5e-8f56-485f-b9ec-201de6f42f88 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:09.472844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93699b5e-8f56-485f-b9ec-201de6f42f88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:09.473582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93699b5e-8f56-485f-b9ec-201de6f42f88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:09.477683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93699b5e-8f56-485f-b9ec-201de6f42f88 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:09.488011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 125/252] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:09.489110 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93699b5e-8f56-485f-b9ec-201de6f42f88 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:09.613932 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93699b5e-8f56-485f-b9ec-201de6f42f88 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:09.627494 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a6566-5b3e-421b-91a9-a9f967f2435f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:09.652494 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:09.668343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae6b2d-4ecb-4301-91a3-53e973a90e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:09.669069 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae6b2d-4ecb-4301-91a3-53e973a90e06 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:09.669361 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae6b2d-4ecb-4301-91a3-53e973a90e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:09.669645 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae6b2d-4ecb-4301-91a3-53e973a90e06 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:09.670413 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae6b2d-4ecb-4301-91a3-53e973a90e06 None admin] RBAC: Adding query filter params (name=member) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:09.670727 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae6b2d-4ecb-4301-91a3-53e973a90e06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:09.673182 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae6b2d-4ecb-4301-91a3-53e973a90e06 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:09.687423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb3533-dfa3-4158-9984-8c854f271fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:09.688139 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb3533-dfa3-4158-9984-8c854f271fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:09.688433 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb3533-dfa3-4158-9984-8c854f271fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:09.688714 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb3533-dfa3-4158-9984-8c854f271fc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:09.689567 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb3533-dfa3-4158-9984-8c854f271fc9 None admin] RBAC: Adding query filter params (role.id=fd276ad89f2d4192b4f90e126d9362d3, scope.project.id=7bfe28b4f0f648dd906926f955192736, user.id=f30246b5afe84bdeaee3960a3bd3117c) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:09.689878 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb3533-dfa3-4158-9984-8c854f271fc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:09.691386 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb3533-dfa3-4158-9984-8c854f271fc9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:09.694931 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 126/253] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:55:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:09.697167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae6b2d-4ecb-4301-91a3-53e973a90e06 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:09.705190 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 128/254] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:55:09 2022] GET /identity/v3/role_assignments?user.id=f30246b5afe84bdeaee3960a3bd3117c&scope.project.id=7bfe28b4f0f648dd906926f955192736&role.id=fd276ad89f2d4192b4f90e126d9362d3 => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:09.708349 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb3533-dfa3-4158-9984-8c854f271fc9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:09.806391 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:09.916210 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae6b2d-4ecb-4301-91a3-53e973a90e06 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:09.984325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87638a67-82de-436f-a91d-507bcf9762ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:09.985292 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87638a67-82de-436f-a91d-507bcf9762ce None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:09.985586 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87638a67-82de-436f-a91d-507bcf9762ce None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:09.985869 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87638a67-82de-436f-a91d-507bcf9762ce None admin] PATH_INFO: `/v3/users/49eedf7e18eb4505889fa1b78da2b449` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:09.995969 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87638a67-82de-436f-a91d-507bcf9762ce None admin] RBAC: Authorizing `identity:get_user(user_id=49eedf7e18eb4505889fa1b78da2b449)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:09.995969 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87638a67-82de-436f-a91d-507bcf9762ce None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:09.995969 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 127/255] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:55:09 2022] GET /identity/v3/users/49eedf7e18eb4505889fa1b78da2b449 => generated 283 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:10.000914 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87638a67-82de-436f-a91d-507bcf9762ce None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:10.026454 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb3533-dfa3-4158-9984-8c854f271fc9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:10.074365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:10.097218 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b781f16-46ee-480e-99c9-d2c73f18ed4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:10.098223 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b781f16-46ee-480e-99c9-d2c73f18ed4a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:10.098516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b781f16-46ee-480e-99c9-d2c73f18ed4a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:10.098857 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b781f16-46ee-480e-99c9-d2c73f18ed4a None admin] PATH_INFO: `/v3/users/7867709f132b48fe9721420e8f43a191` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:10.104676 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b781f16-46ee-480e-99c9-d2c73f18ed4a None admin] RBAC: Authorizing `identity:get_user(user_id=7867709f132b48fe9721420e8f43a191)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:10.106452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b781f16-46ee-480e-99c9-d2c73f18ed4a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:10.108191 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 129/256] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:55:09 2022] GET /identity/v3/users/7867709f132b48fe9721420e8f43a191 => generated 297 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:10.113545 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b781f16-46ee-480e-99c9-d2c73f18ed4a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:10.145401 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87638a67-82de-436f-a91d-507bcf9762ce None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:10.194269 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0666a7-fa27-4a82-9b65-5bafac4af26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:10.195663 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0666a7-fa27-4a82-9b65-5bafac4af26c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:10.196909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0666a7-fa27-4a82-9b65-5bafac4af26c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:10.197394 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0666a7-fa27-4a82-9b65-5bafac4af26c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:10.338962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b781f16-46ee-480e-99c9-d2c73f18ed4a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:10.358901 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:10.363047 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0666a7-fa27-4a82-9b65-5bafac4af26c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:10.364591 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0666a7-fa27-4a82-9b65-5bafac4af26c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:10.390028 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eefac9-7514-4427-a918-28e55489ce58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:10.390913 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eefac9-7514-4427-a918-28e55489ce58 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:10.391202 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eefac9-7514-4427-a918-28e55489ce58 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:10.391482 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eefac9-7514-4427-a918-28e55489ce58 None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:10.392971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eefac9-7514-4427-a918-28e55489ce58 None admin] RBAC: Authorizing `identity:get_project(project_id=7bfe28b4f0f648dd906926f955192736)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:10.395701 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eefac9-7514-4427-a918-28e55489ce58 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:10.398653 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 130/257] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:10 2022] GET /identity/v3/projects/7bfe28b4f0f648dd906926f955192736 => generated 302 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:10.401913 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f5d84-a0f4-400c-b1de-b3cb3e17d689 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:10.402204 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f5d84-a0f4-400c-b1de-b3cb3e17d689 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:10.402491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f5d84-a0f4-400c-b1de-b3cb3e17d689 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:10.403753 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 131/258] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:10.409116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f5d84-a0f4-400c-b1de-b3cb3e17d689 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:10.476740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:10.595842 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 128/259] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:10.597833 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff7639-76b5-46a4-8c35-b643fdc160cb None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:10.598131 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff7639-76b5-46a4-8c35-b643fdc160cb None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:10.598412 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff7639-76b5-46a4-8c35-b643fdc160cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:10.607331 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f5d84-a0f4-400c-b1de-b3cb3e17d689 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:10.629504 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-efff7639-76b5-46a4-8c35-b643fdc160cb None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:10.632106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-efff7639-76b5-46a4-8c35-b643fdc160cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:10.667465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de7fc9-a29c-4d70-9d86-f0172d66e921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:10.668391 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de7fc9-a29c-4d70-9d86-f0172d66e921 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:10.668681 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de7fc9-a29c-4d70-9d86-f0172d66e921 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:10.668963 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de7fc9-a29c-4d70-9d86-f0172d66e921 None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736/users/7867709f132b48fe9721420e8f43a191/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:10.674590 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de7fc9-a29c-4d70-9d86-f0172d66e921 None admin] RBAC: Authorizing `identity:create_grant(project_id=7bfe28b4f0f648dd906926f955192736, user_id=7867709f132b48fe9721420e8f43a191, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:10.676071 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de7fc9-a29c-4d70-9d86-f0172d66e921 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:10.696527 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 132/260] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:55:10 2022] PUT /identity/v3/projects/7bfe28b4f0f648dd906926f955192736/users/7867709f132b48fe9721420e8f43a191/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:55:10.697346 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de7fc9-a29c-4d70-9d86-f0172d66e921 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:10.765105 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 129/261] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:10.783700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:10.808782 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff7639-76b5-46a4-8c35-b643fdc160cb None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:10.875178 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de7fc9-a29c-4d70-9d86-f0172d66e921 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:10.940746 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75fc39-39fa-41ce-815e-bc0929145f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:10.941701 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75fc39-39fa-41ce-815e-bc0929145f06 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:10.941994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75fc39-39fa-41ce-815e-bc0929145f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:10.942276 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75fc39-39fa-41ce-815e-bc0929145f06 None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:10.944660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75fc39-39fa-41ce-815e-bc0929145f06 None admin] RBAC: Authorizing `identity:get_project(project_id=7bfe28b4f0f648dd906926f955192736)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:10.946778 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75fc39-39fa-41ce-815e-bc0929145f06 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:10.948815 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 133/262] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:10 2022] GET /identity/v3/projects/7bfe28b4f0f648dd906926f955192736 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:10.960019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75fc39-39fa-41ce-815e-bc0929145f06 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:10.976336 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:11.027071 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff7639-76b5-46a4-8c35-b643fdc160cb None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:11.067956 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ce9e8-261c-4129-ad67-5efcf55cdc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:11.068848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ce9e8-261c-4129-ad67-5efcf55cdc89 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:11.069148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ce9e8-261c-4129-ad67-5efcf55cdc89 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:11.069554 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ce9e8-261c-4129-ad67-5efcf55cdc89 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:11.081514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ce9e8-261c-4129-ad67-5efcf55cdc89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:11.090593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-f08ce9e8-261c-4129-ad67-5efcf55cdc89 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:55:11.090593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:11.090593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:11.090593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:11.090593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:11.090593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:11.090593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:11.090593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:11.090593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:11.090593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:11.090593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:11.090593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:11.090593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:11.090593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:11.090593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:11.090593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:11.091502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:11.091502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:11.091502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:11.091502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:11.091502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:11.091502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:11.091502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:11.091502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:11.091502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:11.091502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:11.091502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:11.091502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:11.091502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:11.091502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:11.091502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:11.091502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:11.092340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:11.092340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:11.092340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:11.092340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:11.092340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:11.092340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:11.092340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:11.092340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:11.092340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:11.092340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:11.092340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:55:11.092340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:11.092340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 130/263] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Jul 18 13:55:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:55:11.095555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75fc39-39fa-41ce-815e-bc0929145f06 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:11.098230 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ce9e8-261c-4129-ad67-5efcf55cdc89 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:11.103471 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:11.134259 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1ae3d-65ec-4190-bf6a-e793cf8ff9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:11.134850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1ae3d-65ec-4190-bf6a-e793cf8ff9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:11.135136 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1ae3d-65ec-4190-bf6a-e793cf8ff9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:11.135414 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1ae3d-65ec-4190-bf6a-e793cf8ff9d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:11.136294 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef1ae3d-65ec-4190-bf6a-e793cf8ff9d2 None admin] RBAC: Adding query filter params (role.id=c60a99de086f45c6820f7872a6b43feb, scope.project.id=7bfe28b4f0f648dd906926f955192736, user.id=49eedf7e18eb4505889fa1b78da2b449) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:11.136612 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef1ae3d-65ec-4190-bf6a-e793cf8ff9d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:11.138255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef1ae3d-65ec-4190-bf6a-e793cf8ff9d2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:11.144838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 134/264] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:55:10 2022] GET /identity/v3/role_assignments?user.id=49eedf7e18eb4505889fa1b78da2b449&scope.project.id=7bfe28b4f0f648dd906926f955192736&role.id=c60a99de086f45c6820f7872a6b43feb => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:11.247985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ce9e8-261c-4129-ad67-5efcf55cdc89 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:11.294255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076edc8-e68b-4186-b7e2-cea59fd5ed64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:11.295024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076edc8-e68b-4186-b7e2-cea59fd5ed64 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:11.295320 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076edc8-e68b-4186-b7e2-cea59fd5ed64 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:11.295611 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076edc8-e68b-4186-b7e2-cea59fd5ed64 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:11.296455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076edc8-e68b-4186-b7e2-cea59fd5ed64 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:11.296776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076edc8-e68b-4186-b7e2-cea59fd5ed64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:11.298236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076edc8-e68b-4186-b7e2-cea59fd5ed64 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:11.304301 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:11.307964 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 131/265] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:11.313883 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1ae3d-65ec-4190-bf6a-e793cf8ff9d2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:11.445549 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1ae3d-65ec-4190-bf6a-e793cf8ff9d2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:11.483357 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b05b6-b2c2-4792-ba5b-7eac87bed104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:11.484298 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b05b6-b2c2-4792-ba5b-7eac87bed104 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:11.484582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b05b6-b2c2-4792-ba5b-7eac87bed104 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:11.484857 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b05b6-b2c2-4792-ba5b-7eac87bed104 None admin] PATH_INFO: `/v3/users/49eedf7e18eb4505889fa1b78da2b449` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:11.486069 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b05b6-b2c2-4792-ba5b-7eac87bed104 None admin] RBAC: Authorizing `identity:get_user(user_id=49eedf7e18eb4505889fa1b78da2b449)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:11.487536 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b05b6-b2c2-4792-ba5b-7eac87bed104 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:11.489521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 135/266] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:55:11 2022] GET /identity/v3/users/49eedf7e18eb4505889fa1b78da2b449 => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:11.494757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076edc8-e68b-4186-b7e2-cea59fd5ed64 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:11.603826 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076edc8-e68b-4186-b7e2-cea59fd5ed64 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:11.640163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b94c6-6156-489c-8465-a28a9ebb4d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:11.641112 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b94c6-6156-489c-8465-a28a9ebb4d32 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:11.641597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b94c6-6156-489c-8465-a28a9ebb4d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:11.642057 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b94c6-6156-489c-8465-a28a9ebb4d32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:11.709880 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:11.716276 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b94c6-6156-489c-8465-a28a9ebb4d32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:11.717771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b94c6-6156-489c-8465-a28a9ebb4d32 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:11.833441 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 132/267] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:11.837592 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805612c8-feec-49bb-8012-e6b5093b1a77 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:11.838747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805612c8-feec-49bb-8012-e6b5093b1a77 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:11.839053 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805612c8-feec-49bb-8012-e6b5093b1a77 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:11.839152 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b94c6-6156-489c-8465-a28a9ebb4d32 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:11.844377 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 136/268] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:11.845882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c02c40-b7f9-4ae8-ad3d-a1ef74a382d2 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:11.846379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c02c40-b7f9-4ae8-ad3d-a1ef74a382d2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:11.846861 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c02c40-b7f9-4ae8-ad3d-a1ef74a382d2 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:11.849105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 137/269] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:11.852618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7434489-6f31-4bcd-9c36-08d4d2c8dfdc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:11.853366 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7434489-6f31-4bcd-9c36-08d4d2c8dfdc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:11.853876 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7434489-6f31-4bcd-9c36-08d4d2c8dfdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:11.878290 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c7434489-6f31-4bcd-9c36-08d4d2c8dfdc None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:11.879883 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c7434489-6f31-4bcd-9c36-08d4d2c8dfdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:11.919930 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:11.943600 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:11.954234 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b94c6-6156-489c-8465-a28a9ebb4d32 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:11.988113 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 138/270] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:11.989641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270ad94-a2b1-4994-962d-9cdead2dc35e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:11.990066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270ad94-a2b1-4994-962d-9cdead2dc35e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:11.990472 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270ad94-a2b1-4994-962d-9cdead2dc35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:11.999403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a650fd17-7199-4b35-8220-40885bb3a386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:12.000485 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650fd17-7199-4b35-8220-40885bb3a386 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:12.000919 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650fd17-7199-4b35-8220-40885bb3a386 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:12.001343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650fd17-7199-4b35-8220-40885bb3a386 None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:12.003420 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a650fd17-7199-4b35-8220-40885bb3a386 None admin] RBAC: Authorizing `identity:get_project(project_id=7bfe28b4f0f648dd906926f955192736)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:12.005081 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a650fd17-7199-4b35-8220-40885bb3a386 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:12.011299 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 133/271] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:11 2022] GET /identity/v3/projects/7bfe28b4f0f648dd906926f955192736 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:12.012360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a650fd17-7199-4b35-8220-40885bb3a386 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:12.020393 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4270ad94-a2b1-4994-962d-9cdead2dc35e None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:12.021788 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4270ad94-a2b1-4994-962d-9cdead2dc35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:12.117530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 139/272] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:12.118585 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270ad94-a2b1-4994-962d-9cdead2dc35e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:12.124999 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a650fd17-7199-4b35-8220-40885bb3a386 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:12.171191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b278b-e974-4b1f-9aac-b86ae47bbc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:12.172365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b278b-e974-4b1f-9aac-b86ae47bbc38 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:12.172816 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b278b-e974-4b1f-9aac-b86ae47bbc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:12.173329 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b278b-e974-4b1f-9aac-b86ae47bbc38 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:12.174671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b278b-e974-4b1f-9aac-b86ae47bbc38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:12.176501 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b278b-e974-4b1f-9aac-b86ae47bbc38 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:12.178442 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 134/273] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:12.179391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b278b-e974-4b1f-9aac-b86ae47bbc38 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:12.212550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:12.262788 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:12.267524 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270ad94-a2b1-4994-962d-9cdead2dc35e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:12.287456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b278b-e974-4b1f-9aac-b86ae47bbc38 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:12.305079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987914e-3129-4ce5-bd73-924ba6315aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:12.306114 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987914e-3129-4ce5-bd73-924ba6315aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:12.306621 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987914e-3129-4ce5-bd73-924ba6315aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:12.307110 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987914e-3129-4ce5-bd73-924ba6315aa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:12.308177 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987914e-3129-4ce5-bd73-924ba6315aa1 None admin] RBAC: Adding query filter params (role.id=580aae575f414b3c9b4ccf1e27e0b640, scope.project.id=7bfe28b4f0f648dd906926f955192736, user.id=49eedf7e18eb4505889fa1b78da2b449) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:12.308721 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987914e-3129-4ce5-bd73-924ba6315aa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:12.310374 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987914e-3129-4ce5-bd73-924ba6315aa1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:12.316920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 140/274] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:55:12 2022] GET /identity/v3/role_assignments?user.id=49eedf7e18eb4505889fa1b78da2b449&scope.project.id=7bfe28b4f0f648dd906926f955192736&role.id=580aae575f414b3c9b4ccf1e27e0b640 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:12.317944 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987914e-3129-4ce5-bd73-924ba6315aa1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:12.323283 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8555fb-45ad-408f-ae81-ea88a36e4420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:12.324132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8555fb-45ad-408f-ae81-ea88a36e4420 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:12.324423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8555fb-45ad-408f-ae81-ea88a36e4420 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:12.324709 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8555fb-45ad-408f-ae81-ea88a36e4420 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:12.337391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8555fb-45ad-408f-ae81-ea88a36e4420 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:12.345456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-aa8555fb-45ad-408f-ae81-ea88a36e4420 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:12.345456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:12.345456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:12.345456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:12.345456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:12.345456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:12.345456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:12.345456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:12.345456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:12.345456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:12.345456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:12.345456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:12.345456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:12.345456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:12.345456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:12.345456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:12.346390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:12.346390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:12.346390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:12.346390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:12.346390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:12.346390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:12.346390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:12.346390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:12.346390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:12.346390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:12.346390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:12.346390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:12.346390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:12.346390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:12.346390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:12.346390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:12.347198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:12.347198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:12.347198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:12.347198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:12.347198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:12.347198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:12.347198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:12.347198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:12.347198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:12.347198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:12.347198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:12.347198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:12.347198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 135/275] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:12.353456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8555fb-45ad-408f-ae81-ea88a36e4420 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:12.452372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987914e-3129-4ce5-bd73-924ba6315aa1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:12.466750 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:12.480643 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8555fb-45ad-408f-ae81-ea88a36e4420 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:12.501048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd72be-fbe0-4a2d-a3c7-204a74d95d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:12.502117 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd72be-fbe0-4a2d-a3c7-204a74d95d3b None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:12.502607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd72be-fbe0-4a2d-a3c7-204a74d95d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:12.503085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd72be-fbe0-4a2d-a3c7-204a74d95d3b None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:12.504051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd72be-fbe0-4a2d-a3c7-204a74d95d3b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:12.506179 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd72be-fbe0-4a2d-a3c7-204a74d95d3b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:12.535577 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 141/276] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Jul 18 13:55:12 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:55:12.559144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b986aac6-6517-4439-8124-77885f844aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:12.560816 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986aac6-6517-4439-8124-77885f844aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:12.561479 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986aac6-6517-4439-8124-77885f844aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:12.562029 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986aac6-6517-4439-8124-77885f844aa7 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:12.564204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b986aac6-6517-4439-8124-77885f844aa7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:12.564809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b986aac6-6517-4439-8124-77885f844aa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:12.566609 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b986aac6-6517-4439-8124-77885f844aa7 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:12.578965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 136/277] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:55:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:12.584358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd72be-fbe0-4a2d-a3c7-204a74d95d3b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:12.609809 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:12.644977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe38b6-ebff-423f-ade1-38fe561fe6a9 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:12.645606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe38b6-ebff-423f-ade1-38fe561fe6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:12.646113 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe38b6-ebff-423f-ade1-38fe561fe6a9 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:12.648156 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 137/278] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:12.654059 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65b0d1-1cfc-4445-9879-c4ef3087329a None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:12.654673 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65b0d1-1cfc-4445-9879-c4ef3087329a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:12.655156 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65b0d1-1cfc-4445-9879-c4ef3087329a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:12.679527 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-9e65b0d1-1cfc-4445-9879-c4ef3087329a None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:12.681776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9e65b0d1-1cfc-4445-9879-c4ef3087329a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:12.706252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd72be-fbe0-4a2d-a3c7-204a74d95d3b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:12.742841 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e15ec-6497-4d83-87b9-f5d24528790d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:12.744316 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e15ec-6497-4d83-87b9-f5d24528790d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:12.745230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e15ec-6497-4d83-87b9-f5d24528790d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:12.745678 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e15ec-6497-4d83-87b9-f5d24528790d None admin] PATH_INFO: `/v3/users/e02fde0d54cd4f09a8f614bcf20be44b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:12.763676 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e15ec-6497-4d83-87b9-f5d24528790d None admin] RBAC: Authorizing `identity:get_user(user_id=e02fde0d54cd4f09a8f614bcf20be44b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:12.764461 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 138/279] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 13:55:12.765722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e15ec-6497-4d83-87b9-f5d24528790d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:12.767622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 142/280] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:55:12 2022] GET /identity/v3/users/e02fde0d54cd4f09a8f614bcf20be44b => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:12.773214 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65b0d1-1cfc-4445-9879-c4ef3087329a None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:12.793093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e15ec-6497-4d83-87b9-f5d24528790d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:12.876717 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:12.895695 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:12.904566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65b0d1-1cfc-4445-9879-c4ef3087329a None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:12.929956 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e15ec-6497-4d83-87b9-f5d24528790d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:12.959681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653ab53-fa53-4d97-ad0f-bc804f35851c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:12.960381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653ab53-fa53-4d97-ad0f-bc804f35851c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:12.960668 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653ab53-fa53-4d97-ad0f-bc804f35851c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:12.960939 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653ab53-fa53-4d97-ad0f-bc804f35851c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:12.962495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653ab53-fa53-4d97-ad0f-bc804f35851c None admin] RBAC: Adding query filter params (role.id=1399a838106b4f9bb5fa1c942cc3ef22, user.id=e02fde0d54cd4f09a8f614bcf20be44b, scope.system=all) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:12.962808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653ab53-fa53-4d97-ad0f-bc804f35851c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:12.964314 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653ab53-fa53-4d97-ad0f-bc804f35851c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:12.970783 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 139/281] 10.222.0.94 () {56 vars in 1308 bytes} [Mon Jul 18 13:55:12 2022] GET /identity/v3/role_assignments?user.id=e02fde0d54cd4f09a8f614bcf20be44b&scope.system=all&role.id=1399a838106b4f9bb5fa1c942cc3ef22 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:12.983640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591fc6f4-fb40-42b3-9350-271b98e58298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:12.984512 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fc6f4-fb40-42b3-9350-271b98e58298 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:12.984806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fc6f4-fb40-42b3-9350-271b98e58298 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:12.985092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591fc6f4-fb40-42b3-9350-271b98e58298 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:12.999565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591fc6f4-fb40-42b3-9350-271b98e58298 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:13.007805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-591fc6f4-fb40-42b3-9350-271b98e58298 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 13:55:13.007805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:13.007805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:13.007805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:13.007805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:13.007805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:13.007805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:13.007805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:13.007805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:13.007805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:13.007805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:13.007805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:13.007805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:13.007805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:13.007805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:13.007805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:13.008981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:13.008981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:13.008981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:13.008981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:13.008981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:13.008981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:13.008981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:13.008981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:13.008981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:13.008981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:13.008981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:13.008981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:13.008981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:13.008981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:13.008981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:13.008981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:13.009986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:13.009986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:13.009986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:13.009986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:13.009986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:13.009986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:13.009986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:13.009986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:13.009986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:13.009986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:13.009986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 13:55:13.009986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:13.012178 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 143/282] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:55:13.018311 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653ab53-fa53-4d97-ad0f-bc804f35851c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:13.169209 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653ab53-fa53-4d97-ad0f-bc804f35851c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:13.210508 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0adbb-ec00-42a9-94c8-e68737e56167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:13.211192 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0adbb-ec00-42a9-94c8-e68737e56167 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:13.211483 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0adbb-ec00-42a9-94c8-e68737e56167 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:13.211816 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0adbb-ec00-42a9-94c8-e68737e56167 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:13.213085 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0adbb-ec00-42a9-94c8-e68737e56167 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:13.213399 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0adbb-ec00-42a9-94c8-e68737e56167 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:13.214770 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0adbb-ec00-42a9-94c8-e68737e56167 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:13.227740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 140/283] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:55:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:13.234209 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591fc6f4-fb40-42b3-9350-271b98e58298 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:13.395660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591fc6f4-fb40-42b3-9350-271b98e58298 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:13.444738 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda0c4c-67b0-4cac-86e1-cbad06b726ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:13.445550 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda0c4c-67b0-4cac-86e1-cbad06b726ed None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:13.445840 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda0c4c-67b0-4cac-86e1-cbad06b726ed None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:13.446145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda0c4c-67b0-4cac-86e1-cbad06b726ed None admin] PATH_INFO: `/v3/users/f30246b5afe84bdeaee3960a3bd3117c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:13.447983 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda0c4c-67b0-4cac-86e1-cbad06b726ed None admin] RBAC: Authorizing `identity:get_user(user_id=f30246b5afe84bdeaee3960a3bd3117c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:13.449466 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda0c4c-67b0-4cac-86e1-cbad06b726ed None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:13.451113 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 144/284] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:55:13 2022] GET /identity/v3/users/f30246b5afe84bdeaee3960a3bd3117c => generated 297 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:13.457175 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0adbb-ec00-42a9-94c8-e68737e56167 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:13.482176 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:13.513407 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c80cac-9133-48c7-a37f-e1fbe73ae200 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:13.513728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c80cac-9133-48c7-a37f-e1fbe73ae200 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:13.513986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c80cac-9133-48c7-a37f-e1fbe73ae200 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:13.515684 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 145/285] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:55:13.534851 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045524c-3996-4b15-8c8f-b813cfa47141 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:13.535142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045524c-3996-4b15-8c8f-b813cfa47141 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:13.535425 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045524c-3996-4b15-8c8f-b813cfa47141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:13.541596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:13.562697 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5045524c-3996-4b15-8c8f-b813cfa47141 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:13.565246 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5045524c-3996-4b15-8c8f-b813cfa47141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:13.625760 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0adbb-ec00-42a9-94c8-e68737e56167 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:13.674142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29542a-5775-41cd-bf86-9a0c8fc6533b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:13.675364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29542a-5775-41cd-bf86-9a0c8fc6533b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:13.675880 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29542a-5775-41cd-bf86-9a0c8fc6533b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:13.676431 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29542a-5775-41cd-bf86-9a0c8fc6533b None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:13.677819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29542a-5775-41cd-bf86-9a0c8fc6533b None admin] RBAC: Authorizing `identity:get_project(project_id=7bfe28b4f0f648dd906926f955192736)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:13.679457 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29542a-5775-41cd-bf86-9a0c8fc6533b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:13.681949 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 141/286] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:13 2022] GET /identity/v3/projects/7bfe28b4f0f648dd906926f955192736 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:13.689233 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29542a-5775-41cd-bf86-9a0c8fc6533b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:13.709770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 146/287] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 13:55:13.739289 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045524c-3996-4b15-8c8f-b813cfa47141 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:13.828313 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:13.858597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29542a-5775-41cd-bf86-9a0c8fc6533b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:13.900178 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6932a8f5-a2a9-49a1-8898-0d5f968f55d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:13.901354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932a8f5-a2a9-49a1-8898-0d5f968f55d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:13.901778 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932a8f5-a2a9-49a1-8898-0d5f968f55d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:13.902192 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932a8f5-a2a9-49a1-8898-0d5f968f55d6 None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736/users/f30246b5afe84bdeaee3960a3bd3117c/roles/fd276ad89f2d4192b4f90e126d9362d3` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:13.904388 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6932a8f5-a2a9-49a1-8898-0d5f968f55d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=7bfe28b4f0f648dd906926f955192736, user_id=f30246b5afe84bdeaee3960a3bd3117c, role_id=fd276ad89f2d4192b4f90e126d9362d3)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:13.906599 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6932a8f5-a2a9-49a1-8898-0d5f968f55d6 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:13.921603 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 142/288] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:55:13 2022] PUT /identity/v3/projects/7bfe28b4f0f648dd906926f955192736/users/f30246b5afe84bdeaee3960a3bd3117c/roles/fd276ad89f2d4192b4f90e126d9362d3 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 13:55:13.938962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045524c-3996-4b15-8c8f-b813cfa47141 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:13.982524 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e57d7-da1e-4183-b328-bd9d53b9ebfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:13.983656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e57d7-da1e-4183-b328-bd9d53b9ebfa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:13.984112 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e57d7-da1e-4183-b328-bd9d53b9ebfa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:13.984573 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e57d7-da1e-4183-b328-bd9d53b9ebfa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:13.996347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:14.003167 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e57d7-da1e-4183-b328-bd9d53b9ebfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:14.014796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-939e57d7-da1e-4183-b328-bd9d53b9ebfa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:14.014796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:14.014796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:14.014796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:14.014796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:14.014796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:14.014796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:14.014796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:14.014796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:14.014796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:14.014796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:14.014796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:14.014796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:14.014796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:14.014796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:14.014796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:14.015882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:14.015882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:14.015882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:14.015882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:14.015882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:14.015882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:14.015882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:14.015882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:14.015882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:14.015882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:14.015882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:14.015882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:14.015882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:14.015882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:14.015882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:14.015882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:14.016893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:14.016893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:14.016893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:14.016893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:14.016893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:14.016893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:14.016893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:14.016893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:14.016893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:14.016893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:14.016893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:14.016893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:14.018966 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 147/289] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:55:14.025230 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6932a8f5-a2a9-49a1-8898-0d5f968f55d6 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:14.161442 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6932a8f5-a2a9-49a1-8898-0d5f968f55d6 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:14.198069 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee68558-b3f6-4128-8f07-6fa60948dddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:14.199025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee68558-b3f6-4128-8f07-6fa60948dddb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:14.199455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee68558-b3f6-4128-8f07-6fa60948dddb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:14.199872 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee68558-b3f6-4128-8f07-6fa60948dddb None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:14.200792 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee68558-b3f6-4128-8f07-6fa60948dddb None admin] RBAC: Adding query filter params (name=member) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:14.201270 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee68558-b3f6-4128-8f07-6fa60948dddb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:14.202891 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee68558-b3f6-4128-8f07-6fa60948dddb None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:14.206021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:14.213212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 143/290] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:55:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:14.221711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e57d7-da1e-4183-b328-bd9d53b9ebfa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:14.367982 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e57d7-da1e-4183-b328-bd9d53b9ebfa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:14.406615 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9554c-d92d-4882-85bf-7d926dabba05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:14.407566 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9554c-d92d-4882-85bf-7d926dabba05 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:14.407863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9554c-d92d-4882-85bf-7d926dabba05 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:14.408201 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9554c-d92d-4882-85bf-7d926dabba05 None admin] PATH_INFO: `/v3/users/7867709f132b48fe9721420e8f43a191` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:14.409436 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e9554c-d92d-4882-85bf-7d926dabba05 None admin] RBAC: Authorizing `identity:get_user(user_id=7867709f132b48fe9721420e8f43a191)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:14.410984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e9554c-d92d-4882-85bf-7d926dabba05 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:14.412894 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 148/291] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:55:14 2022] GET /identity/v3/users/7867709f132b48fe9721420e8f43a191 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:14.418383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee68558-b3f6-4128-8f07-6fa60948dddb None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:14.532725 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee68558-b3f6-4128-8f07-6fa60948dddb None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:14.580703 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dfebe8-dd91-474d-a9da-aca802526954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:14.581377 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfebe8-dd91-474d-a9da-aca802526954 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:14.581668 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfebe8-dd91-474d-a9da-aca802526954 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:14.581955 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfebe8-dd91-474d-a9da-aca802526954 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:14.719513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:14.729042 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dfebe8-dd91-474d-a9da-aca802526954 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:14.730543 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dfebe8-dd91-474d-a9da-aca802526954 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:14.881011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 144/292] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:14.886791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9554c-d92d-4882-85bf-7d926dabba05 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:15.016391 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:15.079975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9554c-d92d-4882-85bf-7d926dabba05 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:15.139478 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0c91a-5844-457d-a421-3912f725595e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:15.140380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0c91a-5844-457d-a421-3912f725595e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:15.140675 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0c91a-5844-457d-a421-3912f725595e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:15.140954 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0c91a-5844-457d-a421-3912f725595e None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:15.144680 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f0c91a-5844-457d-a421-3912f725595e None admin] RBAC: Authorizing `identity:get_project(project_id=7bfe28b4f0f648dd906926f955192736)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:15.146173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f0c91a-5844-457d-a421-3912f725595e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:15.148224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 149/293] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:14 2022] GET /identity/v3/projects/7bfe28b4f0f648dd906926f955192736 => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:15.154126 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dfebe8-dd91-474d-a9da-aca802526954 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:15.169767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29602a95-d82a-41c8-8dd7-27c977137585 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:15.170062 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29602a95-d82a-41c8-8dd7-27c977137585 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:15.170339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29602a95-d82a-41c8-8dd7-27c977137585 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:15.173003 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 150/294] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:15.177333 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0eed0-ea40-405f-bf5e-fc21e3dfff1e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:15.177632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0eed0-ea40-405f-bf5e-fc21e3dfff1e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:15.177921 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0eed0-ea40-405f-bf5e-fc21e3dfff1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:15.195314 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-35a0eed0-ea40-405f-bf5e-fc21e3dfff1e None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:15.196615 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-35a0eed0-ea40-405f-bf5e-fc21e3dfff1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:15.308051 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:15.315965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dfebe8-dd91-474d-a9da-aca802526954 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:15.327970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 151/295] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:15.327970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe36b22-2337-4b39-ab0d-f2f8fe86ee55 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:15.327970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe36b22-2337-4b39-ab0d-f2f8fe86ee55 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:15.327970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe36b22-2337-4b39-ab0d-f2f8fe86ee55 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:15.331965 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 152/296] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:15.343964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaee24c-630a-4699-a73b-d37a94494bcd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:15.343964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaee24c-630a-4699-a73b-d37a94494bcd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:15.343964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaee24c-630a-4699-a73b-d37a94494bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:15.369163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bfaee24c-630a-4699-a73b-d37a94494bcd None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:15.374154 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bfaee24c-630a-4699-a73b-d37a94494bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:15.381587 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5efdc8-a68d-42f8-bfe2-0704dd320961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:15.382270 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5efdc8-a68d-42f8-bfe2-0704dd320961 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:15.387965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5efdc8-a68d-42f8-bfe2-0704dd320961 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:15.387965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5efdc8-a68d-42f8-bfe2-0704dd320961 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:15.387965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5efdc8-a68d-42f8-bfe2-0704dd320961 None admin] RBAC: Adding query filter params (role.id=1399a838106b4f9bb5fa1c942cc3ef22, scope.project.id=7bfe28b4f0f648dd906926f955192736, user.id=7867709f132b48fe9721420e8f43a191) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:15.387965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5efdc8-a68d-42f8-bfe2-0704dd320961 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:15.387965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5efdc8-a68d-42f8-bfe2-0704dd320961 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:15.395775 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 145/297] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:55:15 2022] GET /identity/v3/role_assignments?user.id=7867709f132b48fe9721420e8f43a191&scope.project.id=7bfe28b4f0f648dd906926f955192736&role.id=1399a838106b4f9bb5fa1c942cc3ef22 => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:15.397227 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5efdc8-a68d-42f8-bfe2-0704dd320961 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:15.499977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:15.518536 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 153/298] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:15.554939 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaee24c-630a-4699-a73b-d37a94494bcd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:15.630920 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5efdc8-a68d-42f8-bfe2-0704dd320961 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:15.692603 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280807d3-e7af-44c9-bbf6-c6b692c49349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:15.693423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280807d3-e7af-44c9-bbf6-c6b692c49349 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:15.693707 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280807d3-e7af-44c9-bbf6-c6b692c49349 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:15.693990 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280807d3-e7af-44c9-bbf6-c6b692c49349 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:15.716605 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280807d3-e7af-44c9-bbf6-c6b692c49349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:15.732326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-280807d3-e7af-44c9-bbf6-c6b692c49349 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 13:55:15.732326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:15.732326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:15.732326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:15.732326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:15.732326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:15.732326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:15.732326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:15.732326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:15.732326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:15.732326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:15.732326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:15.732326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:15.732326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:15.732326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:15.732326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:15.734340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:15.734340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:15.734340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:15.734340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:15.734340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:15.734340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:15.734340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:15.734340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:15.734340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:15.734340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:15.734340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:15.734340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:15.734340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:15.734340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:15.734340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:15.734340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:15.735144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:15.735144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:15.735144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:15.735144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:15.735144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:15.735144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:15.735144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:15.735144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:15.735144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:15.735144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:15.735144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 13:55:15.735144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:15.738336 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 146/299] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:15.745628 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d779438-9bee-4a9e-9d24-bd8f8893ff82 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:15.745998 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d779438-9bee-4a9e-9d24-bd8f8893ff82 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:15.746357 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d779438-9bee-4a9e-9d24-bd8f8893ff82 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:15.747788 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 147/300] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:15.748961 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d779438-9bee-4a9e-9d24-bd8f8893ff82 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:15.749270 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaee24c-630a-4699-a73b-d37a94494bcd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:15.781800 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:15.798385 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009fdd4-6f29-4706-86cb-5c1ed27a343a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:15.799241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009fdd4-6f29-4706-86cb-5c1ed27a343a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:15.799536 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009fdd4-6f29-4706-86cb-5c1ed27a343a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:15.799807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009fdd4-6f29-4706-86cb-5c1ed27a343a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:15.814782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009fdd4-6f29-4706-86cb-5c1ed27a343a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:15.823124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-d009fdd4-6f29-4706-86cb-5c1ed27a343a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:15.823124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:15.823124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:15.823124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:15.823124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:15.823124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:15.823124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:15.823124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:15.823124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:15.823124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:15.823124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:15.823124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:15.823124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:15.823124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:15.823124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:15.823124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:15.824045 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:15.824045 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:15.824045 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:15.824045 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:15.824045 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:15.824045 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:15.824045 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:15.824045 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:15.824045 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:15.824045 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:15.824045 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:15.824045 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:15.824045 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:15.824045 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:15.824045 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:15.824045 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:15.825987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:15.825987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:15.825987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:15.825987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:15.825987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:15.825987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:15.825987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:15.825987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:15.825987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:15.825987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:15.825987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:15.825987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:15.825987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 154/301] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:15.825987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d2b70-1646-4b7c-b39b-b917ad5d7aa9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:15.826693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d2b70-1646-4b7c-b39b-b917ad5d7aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:15.826693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d2b70-1646-4b7c-b39b-b917ad5d7aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:15.855057 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9b0d2b70-1646-4b7c-b39b-b917ad5d7aa9 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:15.860678 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9b0d2b70-1646-4b7c-b39b-b917ad5d7aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:15.888643 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d779438-9bee-4a9e-9d24-bd8f8893ff82 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:15.906422 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:15.924383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1c666-5c21-4db4-a9f4-1d3e4abbb951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:15.925059 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1c666-5c21-4db4-a9f4-1d3e4abbb951 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:15.925354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1c666-5c21-4db4-a9f4-1d3e4abbb951 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:15.925640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1c666-5c21-4db4-a9f4-1d3e4abbb951 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:15.926414 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1c666-5c21-4db4-a9f4-1d3e4abbb951 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:15.926712 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1c666-5c21-4db4-a9f4-1d3e4abbb951 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:15.928132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1c666-5c21-4db4-a9f4-1d3e4abbb951 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:15.928345 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 155/302] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:15.929379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d2b70-1646-4b7c-b39b-b917ad5d7aa9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:15.951521 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 148/303] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:55:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:15.952904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1c666-5c21-4db4-a9f4-1d3e4abbb951 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:16.041087 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d2b70-1646-4b7c-b39b-b917ad5d7aa9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:16.045975 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:16.063989 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1c666-5c21-4db4-a9f4-1d3e4abbb951 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:16.076798 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbfb2e7-ccb9-4b1e-a37d-04977cf1ee41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:16.077468 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfb2e7-ccb9-4b1e-a37d-04977cf1ee41 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:16.077755 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfb2e7-ccb9-4b1e-a37d-04977cf1ee41 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:16.078036 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfb2e7-ccb9-4b1e-a37d-04977cf1ee41 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:16.079038 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbfb2e7-ccb9-4b1e-a37d-04977cf1ee41 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:16.079358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbfb2e7-ccb9-4b1e-a37d-04977cf1ee41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:16.080760 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbfb2e7-ccb9-4b1e-a37d-04977cf1ee41 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:16.090216 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 156/304] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:55:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:16.091000 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbfb2e7-ccb9-4b1e-a37d-04977cf1ee41 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:16.107016 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9a5a1-3e6b-4bc6-bcbf-bbb52bc5fca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:16.108223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9a5a1-3e6b-4bc6-bcbf-bbb52bc5fca1 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:16.108514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9a5a1-3e6b-4bc6-bcbf-bbb52bc5fca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:16.108797 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f9a5a1-3e6b-4bc6-bcbf-bbb52bc5fca1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:16.110157 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9a5a1-3e6b-4bc6-bcbf-bbb52bc5fca1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:16.111769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f9a5a1-3e6b-4bc6-bcbf-bbb52bc5fca1 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:16.117656 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 149/305] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:16.117656 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9a5a1-3e6b-4bc6-bcbf-bbb52bc5fca1 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:16.137088 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:16.199125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbfb2e7-ccb9-4b1e-a37d-04977cf1ee41 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:16.224262 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f9a5a1-3e6b-4bc6-bcbf-bbb52bc5fca1 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:16.233773 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72f336-9907-4513-a42b-80159ed49ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:16.234644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72f336-9907-4513-a42b-80159ed49ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:16.234932 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72f336-9907-4513-a42b-80159ed49ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:16.235207 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72f336-9907-4513-a42b-80159ed49ee1 None admin] PATH_INFO: `/v3/users/ad57f4b94892434db8c7b1e1d491843a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:16.249287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd72f336-9907-4513-a42b-80159ed49ee1 None admin] RBAC: Authorizing `identity:get_user(user_id=ad57f4b94892434db8c7b1e1d491843a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:16.250755 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd72f336-9907-4513-a42b-80159ed49ee1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:16.252662 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 157/306] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:55:16 2022] GET /identity/v3/users/ad57f4b94892434db8c7b1e1d491843a => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:16.253316 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72f336-9907-4513-a42b-80159ed49ee1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:16.266932 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7087961-392e-4125-a41d-de73dfab8683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:16.267810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7087961-392e-4125-a41d-de73dfab8683 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:16.268132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7087961-392e-4125-a41d-de73dfab8683 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:16.268414 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7087961-392e-4125-a41d-de73dfab8683 None admin] PATH_INFO: `/v3/users/e02fde0d54cd4f09a8f614bcf20be44b` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:16.269643 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7087961-392e-4125-a41d-de73dfab8683 None admin] RBAC: Authorizing `identity:get_user(user_id=e02fde0d54cd4f09a8f614bcf20be44b)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:16.271083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7087961-392e-4125-a41d-de73dfab8683 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:16.272885 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 150/307] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:55:16 2022] GET /identity/v3/users/e02fde0d54cd4f09a8f614bcf20be44b => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:16.278010 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7087961-392e-4125-a41d-de73dfab8683 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:16.302489 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:16.355651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:16.358866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72f336-9907-4513-a42b-80159ed49ee1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:16.385327 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7087961-392e-4125-a41d-de73dfab8683 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:16.400034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c47c2e-4fb5-4da6-8166-53b61c164d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:16.401266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c47c2e-4fb5-4da6-8166-53b61c164d5c None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:16.401737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c47c2e-4fb5-4da6-8166-53b61c164d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:16.402022 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c47c2e-4fb5-4da6-8166-53b61c164d5c None admin] PATH_INFO: `/v3/groups` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:16.402723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c47c2e-4fb5-4da6-8166-53b61c164d5c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:16.404208 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c47c2e-4fb5-4da6-8166-53b61c164d5c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:16.427978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 158/308] 10.222.0.94 () {60 vars in 1134 bytes} [Mon Jul 18 13:55:16 2022] POST /identity/v3/groups => generated 229 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:55:16.427978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c47c2e-4fb5-4da6-8166-53b61c164d5c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:16.431517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01b170-c21a-40e8-b0e3-605b8adc37c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:16.432509 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01b170-c21a-40e8-b0e3-605b8adc37c7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:16.432795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01b170-c21a-40e8-b0e3-605b8adc37c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:16.434085 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01b170-c21a-40e8-b0e3-605b8adc37c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:16.435057 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01b170-c21a-40e8-b0e3-605b8adc37c7 None admin] RBAC: Adding query filter params (role.id=fd276ad89f2d4192b4f90e126d9362d3, user.id=ad57f4b94892434db8c7b1e1d491843a, scope.system=all) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:16.435439 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01b170-c21a-40e8-b0e3-605b8adc37c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:16.437015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01b170-c21a-40e8-b0e3-605b8adc37c7 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:16.448161 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 151/309] 10.222.0.94 () {56 vars in 1308 bytes} [Mon Jul 18 13:55:16 2022] GET /identity/v3/role_assignments?user.id=ad57f4b94892434db8c7b1e1d491843a&scope.system=all&role.id=fd276ad89f2d4192b4f90e126d9362d3 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:16.547573 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f304fe-8409-46ca-b1c5-b67ad839b018 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:16.547776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f304fe-8409-46ca-b1c5-b67ad839b018 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:16.548083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f304fe-8409-46ca-b1c5-b67ad839b018 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:16.549864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 152/310] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:55:16.556259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba20253-9741-4382-9388-8c94e4613875 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:16.556531 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba20253-9741-4382-9388-8c94e4613875 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:16.556814 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba20253-9741-4382-9388-8c94e4613875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:16.561188 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c47c2e-4fb5-4da6-8166-53b61c164d5c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:16.575698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-1ba20253-9741-4382-9388-8c94e4613875 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:16.577142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1ba20253-9741-4382-9388-8c94e4613875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:16.603067 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d1fbb-57f3-422a-a09a-35ef625919de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:16.604289 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d1fbb-57f3-422a-a09a-35ef625919de None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:16.605381 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d1fbb-57f3-422a-a09a-35ef625919de None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:16.605831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d1fbb-57f3-422a-a09a-35ef625919de None admin] PATH_INFO: `/v3/system/users/e02fde0d54cd4f09a8f614bcf20be44b/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:16.607482 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51d1fbb-57f3-422a-a09a-35ef625919de None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e02fde0d54cd4f09a8f614bcf20be44b, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:16.608955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51d1fbb-57f3-422a-a09a-35ef625919de None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:16.621161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 159/311] 10.222.0.94 () {58 vars in 1334 bytes} [Mon Jul 18 13:55:16 2022] PUT /identity/v3/system/users/e02fde0d54cd4f09a8f614bcf20be44b/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:55:16.667523 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 153/312] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 13:55:16.690968 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d1fbb-57f3-422a-a09a-35ef625919de None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:16.721909 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:16.809478 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d1fbb-57f3-422a-a09a-35ef625919de None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:16.845238 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068ad09-475b-4497-b11a-2c1feb8360fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:16.846059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068ad09-475b-4497-b11a-2c1feb8360fd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:16.846350 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068ad09-475b-4497-b11a-2c1feb8360fd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:16.846636 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068ad09-475b-4497-b11a-2c1feb8360fd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:16.858248 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068ad09-475b-4497-b11a-2c1feb8360fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:16.866168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-1068ad09-475b-4497-b11a-2c1feb8360fd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 13:55:16.866168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:16.866168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:16.866168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:16.866168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:16.866168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:16.866168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:16.866168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:16.866168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:16.866168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:16.866168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:16.866168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:16.866168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:16.866168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:16.866168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:16.866168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:16.867021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:16.867021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:16.867021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:16.867021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:16.867021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:16.867021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:16.867021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:16.867021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:16.867021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:16.867021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:16.867021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:16.867021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:16.867021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:16.867021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:16.867021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:16.867021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:16.867862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:16.867862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:16.867862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:16.867862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:16.867862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:16.867862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:16.867862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:16.867862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:16.867862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:16.867862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:16.867862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 13:55:16.867862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:16.867862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 160/313] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:16.875051 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba20253-9741-4382-9388-8c94e4613875 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:16.890830 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:16.994226 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba20253-9741-4382-9388-8c94e4613875 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:17.030441 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eae0c5-4820-4aad-93d1-7f53ce9fe6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:17.031125 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eae0c5-4820-4aad-93d1-7f53ce9fe6af None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:17.031413 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eae0c5-4820-4aad-93d1-7f53ce9fe6af None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:17.031695 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eae0c5-4820-4aad-93d1-7f53ce9fe6af None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:17.032473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eae0c5-4820-4aad-93d1-7f53ce9fe6af None admin] RBAC: Adding query filter params (name=reader) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:17.032780 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eae0c5-4820-4aad-93d1-7f53ce9fe6af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:17.034159 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eae0c5-4820-4aad-93d1-7f53ce9fe6af None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:17.043800 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 154/314] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:55:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:17.049065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068ad09-475b-4497-b11a-2c1feb8360fd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:17.137802 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:17.155654 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068ad09-475b-4497-b11a-2c1feb8360fd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:17.191478 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b65959-388a-4ee6-b927-10436240831e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:17.192400 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b65959-388a-4ee6-b927-10436240831e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:17.192692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b65959-388a-4ee6-b927-10436240831e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:17.192975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b65959-388a-4ee6-b927-10436240831e None admin] PATH_INFO: `/v3/users/f30246b5afe84bdeaee3960a3bd3117c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:17.194168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b65959-388a-4ee6-b927-10436240831e None admin] RBAC: Authorizing `identity:get_user(user_id=f30246b5afe84bdeaee3960a3bd3117c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:17.195630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b65959-388a-4ee6-b927-10436240831e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:17.197704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 161/315] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:55:17 2022] GET /identity/v3/users/f30246b5afe84bdeaee3960a3bd3117c => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:17.202964 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eae0c5-4820-4aad-93d1-7f53ce9fe6af None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:17.284882 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:17.309622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eae0c5-4820-4aad-93d1-7f53ce9fe6af None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:17.343522 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c857b2e-9f14-403e-b39e-b9b1c616f4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:17.344208 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c857b2e-9f14-403e-b39e-b9b1c616f4cc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:17.344545 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c857b2e-9f14-403e-b39e-b9b1c616f4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:17.344879 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c857b2e-9f14-403e-b39e-b9b1c616f4cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:17.413212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c857b2e-9f14-403e-b39e-b9b1c616f4cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:17.414603 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c857b2e-9f14-403e-b39e-b9b1c616f4cc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:17.500780 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:17.528770 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 155/316] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:17.533955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b65959-388a-4ee6-b927-10436240831e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:17.639382 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b65959-388a-4ee6-b927-10436240831e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:17.675854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba0906-7623-4e1b-b2c1-44ddfd5009ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:17.677066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba0906-7623-4e1b-b2c1-44ddfd5009ff None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:17.677556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba0906-7623-4e1b-b2c1-44ddfd5009ff None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:17.678061 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba0906-7623-4e1b-b2c1-44ddfd5009ff None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:17.679466 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba0906-7623-4e1b-b2c1-44ddfd5009ff None admin] RBAC: Authorizing `identity:get_project(project_id=7bfe28b4f0f648dd906926f955192736)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:17.681217 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba0906-7623-4e1b-b2c1-44ddfd5009ff None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:17.683491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 162/317] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:17 2022] GET /identity/v3/projects/7bfe28b4f0f648dd906926f955192736 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:17.689142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c857b2e-9f14-403e-b39e-b9b1c616f4cc None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:17.794970 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c857b2e-9f14-403e-b39e-b9b1c616f4cc None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:17.841228 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3824a1-7738-4e4e-a343-808053f9f5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:17.841904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3824a1-7738-4e4e-a343-808053f9f5bc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:17.842193 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3824a1-7738-4e4e-a343-808053f9f5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:17.842479 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3824a1-7738-4e4e-a343-808053f9f5bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:17.843321 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3824a1-7738-4e4e-a343-808053f9f5bc None admin] RBAC: Adding query filter params (role.id=fd276ad89f2d4192b4f90e126d9362d3, scope.project.id=7bfe28b4f0f648dd906926f955192736, user.id=f30246b5afe84bdeaee3960a3bd3117c) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:17.843635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3824a1-7738-4e4e-a343-808053f9f5bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:17.845119 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3824a1-7738-4e4e-a343-808053f9f5bc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:17.851847 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 156/318] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:55:17 2022] GET /identity/v3/role_assignments?user.id=f30246b5afe84bdeaee3960a3bd3117c&scope.project.id=7bfe28b4f0f648dd906926f955192736&role.id=fd276ad89f2d4192b4f90e126d9362d3 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:18.575546 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef25dca-57e5-4469-b11a-e19231f6283c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:18.576085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef25dca-57e5-4469-b11a-e19231f6283c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:18.576161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef25dca-57e5-4469-b11a-e19231f6283c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:18.577685 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 163/319] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:18.582928 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69301231-638f-4263-b77d-9d25cca371a5 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:18.583316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69301231-638f-4263-b77d-9d25cca371a5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:18.583543 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69301231-638f-4263-b77d-9d25cca371a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:18.600346 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-69301231-638f-4263-b77d-9d25cca371a5 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:18.601548 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-69301231-638f-4263-b77d-9d25cca371a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:18.639301 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:18.655545 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb7bc5-61fa-49e7-8946-459e4633e25d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:18.655905 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb7bc5-61fa-49e7-8946-459e4633e25d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:18.656289 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb7bc5-61fa-49e7-8946-459e4633e25d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:18.657732 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 164/320] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:55:18.663176 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8598fd-9717-4b6f-a28d-5676fba90f16 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:18.663470 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8598fd-9717-4b6f-a28d-5676fba90f16 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:18.663745 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8598fd-9717-4b6f-a28d-5676fba90f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:18.674936 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 157/321] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:18.681354 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fa8598fd-9717-4b6f-a28d-5676fba90f16 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:18.682546 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fa8598fd-9717-4b6f-a28d-5676fba90f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:18.697313 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:18.697832 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69301231-638f-4263-b77d-9d25cca371a5 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:18.753107 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 165/322] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 13:55:18.754641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b571f5f-a10a-4a6f-91b2-565af036aa63 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:18.754895 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b571f5f-a10a-4a6f-91b2-565af036aa63 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:18.755113 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b571f5f-a10a-4a6f-91b2-565af036aa63 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:18.756388 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 166/323] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:18.765157 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373efde3-e2b2-4b2f-8691-d759cf46b240 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:18.765450 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373efde3-e2b2-4b2f-8691-d759cf46b240 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:18.765731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373efde3-e2b2-4b2f-8691-d759cf46b240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:18.782678 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-373efde3-e2b2-4b2f-8691-d759cf46b240 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:18.783859 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-373efde3-e2b2-4b2f-8691-d759cf46b240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:18.804019 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69301231-638f-4263-b77d-9d25cca371a5 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:18.839057 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71722154-d8e6-43b0-a55c-5e2c7844363f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:18.839790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71722154-d8e6-43b0-a55c-5e2c7844363f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:18.840114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71722154-d8e6-43b0-a55c-5e2c7844363f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:18.840410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71722154-d8e6-43b0-a55c-5e2c7844363f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:18.847200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 167/324] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 13:55:18.848037 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373efde3-e2b2-4b2f-8691-d759cf46b240 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:18.850752 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71722154-d8e6-43b0-a55c-5e2c7844363f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:18.859052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-71722154-d8e6-43b0-a55c-5e2c7844363f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 13:55:18.859052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:18.859052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:18.859052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:18.859052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:18.859052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:18.859052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:18.859052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:18.859052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:18.859052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:18.859052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:18.859052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:18.859052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:18.859052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:18.859052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:18.859052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:18.859980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:18.859980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:18.859980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:18.859980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:18.859980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:18.859980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:18.859980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:18.859980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:18.859980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:18.859980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:18.859980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:18.859980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:18.859980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:18.859980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:18.859980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:18.859980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:18.860789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:18.860789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:18.860789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:18.860789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:18.860789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:18.860789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:18.860789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:18.860789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:18.860789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:18.860789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:18.860789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 13:55:18.860789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:18.860789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 158/325] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:55:18.869819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71722154-d8e6-43b0-a55c-5e2c7844363f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:18.900222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:18.935555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:18.955663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373efde3-e2b2-4b2f-8691-d759cf46b240 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:18.970871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71722154-d8e6-43b0-a55c-5e2c7844363f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:18.989346 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e69f20-2f5d-4f1f-8658-c20bb1420d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:18.990134 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e69f20-2f5d-4f1f-8658-c20bb1420d26 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:18.990419 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e69f20-2f5d-4f1f-8658-c20bb1420d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:18.990696 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e69f20-2f5d-4f1f-8658-c20bb1420d26 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:18.991858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e69f20-2f5d-4f1f-8658-c20bb1420d26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:18.993910 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e69f20-2f5d-4f1f-8658-c20bb1420d26 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:18.995523 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 168/326] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:18.996371 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e69f20-2f5d-4f1f-8658-c20bb1420d26 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:19.004176 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d43e5-2021-4217-b42d-71cca1ad1a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:19.004846 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d43e5-2021-4217-b42d-71cca1ad1a1e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:19.005129 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d43e5-2021-4217-b42d-71cca1ad1a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:19.005406 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d43e5-2021-4217-b42d-71cca1ad1a1e None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:19.006146 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d43e5-2021-4217-b42d-71cca1ad1a1e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:19.006455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d43e5-2021-4217-b42d-71cca1ad1a1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:19.007816 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d43e5-2021-4217-b42d-71cca1ad1a1e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:19.015784 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 159/327] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:55:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:19.016561 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d43e5-2021-4217-b42d-71cca1ad1a1e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:19.097319 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e69f20-2f5d-4f1f-8658-c20bb1420d26 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:19.107392 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:19.116369 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d43e5-2021-4217-b42d-71cca1ad1a1e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:19.132702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88165570-6665-4479-96b9-533f21f79506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:19.134030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88165570-6665-4479-96b9-533f21f79506 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:19.134331 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88165570-6665-4479-96b9-533f21f79506 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:19.134613 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88165570-6665-4479-96b9-533f21f79506 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:19.145129 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88165570-6665-4479-96b9-533f21f79506 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:19.149327 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007668b-aa41-460c-9b44-472e41a067cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:19.149996 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007668b-aa41-460c-9b44-472e41a067cb None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:19.150284 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007668b-aa41-460c-9b44-472e41a067cb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:19.150562 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007668b-aa41-460c-9b44-472e41a067cb None admin] PATH_INFO: `/v3/groups` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:19.151238 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007668b-aa41-460c-9b44-472e41a067cb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:19.152714 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007668b-aa41-460c-9b44-472e41a067cb None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:19.152961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-88165570-6665-4479-96b9-533f21f79506 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:19.152961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:19.152961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:19.152961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:19.152961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:19.152961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:19.152961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:19.152961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:19.152961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:19.152961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:19.152961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:19.152961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:19.152961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:19.152961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:19.152961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:19.152961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:19.153977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:19.153977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:19.153977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:19.153977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:19.153977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:19.153977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:19.153977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:19.153977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:19.153977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:19.153977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:19.153977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:19.153977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:19.153977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:19.153977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:19.153977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:19.153977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:19.154785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:19.154785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:19.154785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:19.154785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:19.154785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:19.154785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:19.154785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:19.154785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:19.154785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:19.154785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:19.154785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:19.154785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:19.154785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 169/328] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:19.155485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88165570-6665-4479-96b9-533f21f79506 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:19.168561 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 160/329] 10.222.0.94 () {60 vars in 1134 bytes} [Mon Jul 18 13:55:19 2022] POST /identity/v3/groups => generated 226 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:55:19.169373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007668b-aa41-460c-9b44-472e41a067cb None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:19.176398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:19.278597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88165570-6665-4479-96b9-533f21f79506 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:19.298796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007668b-aa41-460c-9b44-472e41a067cb None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:19.316671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088c7f3-d918-4e4f-9fda-cbc3f84325fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:19.317564 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088c7f3-d918-4e4f-9fda-cbc3f84325fc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:19.317845 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088c7f3-d918-4e4f-9fda-cbc3f84325fc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:19.318117 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088c7f3-d918-4e4f-9fda-cbc3f84325fc None admin] PATH_INFO: `/v3/users/ad57f4b94892434db8c7b1e1d491843a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:19.319337 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088c7f3-d918-4e4f-9fda-cbc3f84325fc None admin] RBAC: Authorizing `identity:get_user(user_id=ad57f4b94892434db8c7b1e1d491843a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:19.320822 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088c7f3-d918-4e4f-9fda-cbc3f84325fc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:19.322991 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 170/330] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:55:19 2022] GET /identity/v3/users/ad57f4b94892434db8c7b1e1d491843a => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:19.328524 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088c7f3-d918-4e4f-9fda-cbc3f84325fc None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:19.342526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca2612-e202-4dba-890a-9984bc5a690f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:19.343341 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca2612-e202-4dba-890a-9984bc5a690f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:19.343733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca2612-e202-4dba-890a-9984bc5a690f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:19.344169 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca2612-e202-4dba-890a-9984bc5a690f None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:19.345027 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca2612-e202-4dba-890a-9984bc5a690f None admin] RBAC: Adding query filter params (name=member) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:19.345442 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca2612-e202-4dba-890a-9984bc5a690f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:19.346937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca2612-e202-4dba-890a-9984bc5a690f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:19.355979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 161/331] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:55:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:19.361451 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca2612-e202-4dba-890a-9984bc5a690f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:19.378160 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:19.425082 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:19.464634 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088c7f3-d918-4e4f-9fda-cbc3f84325fc None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:19.486304 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca2612-e202-4dba-890a-9984bc5a690f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:19.507686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e52e72-9e3f-4705-b64b-d981ec73a974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:19.508714 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e52e72-9e3f-4705-b64b-d981ec73a974 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:19.509006 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e52e72-9e3f-4705-b64b-d981ec73a974 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:19.509289 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e52e72-9e3f-4705-b64b-d981ec73a974 None admin] PATH_INFO: `/v3/system/users/ad57f4b94892434db8c7b1e1d491843a/roles/fd276ad89f2d4192b4f90e126d9362d3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:19.510795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e52e72-9e3f-4705-b64b-d981ec73a974 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ad57f4b94892434db8c7b1e1d491843a, role_id=fd276ad89f2d4192b4f90e126d9362d3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:19.512392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e52e72-9e3f-4705-b64b-d981ec73a974 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:19.521740 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 171/332] 10.222.0.94 () {58 vars in 1334 bytes} [Mon Jul 18 13:55:19 2022] PUT /identity/v3/system/users/ad57f4b94892434db8c7b1e1d491843a/roles/fd276ad89f2d4192b4f90e126d9362d3 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 13:55:19.526368 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0df48-45d1-4829-909e-f56059ba8646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:19.527362 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0df48-45d1-4829-909e-f56059ba8646 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:19.527723 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0df48-45d1-4829-909e-f56059ba8646 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:19.528117 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0df48-45d1-4829-909e-f56059ba8646 None admin] PATH_INFO: `/v3/users/e02fde0d54cd4f09a8f614bcf20be44b` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:19.529512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0df48-45d1-4829-909e-f56059ba8646 None admin] RBAC: Authorizing `identity:get_user(user_id=e02fde0d54cd4f09a8f614bcf20be44b)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:19.531078 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0df48-45d1-4829-909e-f56059ba8646 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:19.533199 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 162/333] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:55:19 2022] GET /identity/v3/users/e02fde0d54cd4f09a8f614bcf20be44b => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:19.539480 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e52e72-9e3f-4705-b64b-d981ec73a974 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:19.662196 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e52e72-9e3f-4705-b64b-d981ec73a974 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:19.699489 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27688835-ac39-4d44-bf48-1cdc8cfb341c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:19.700200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27688835-ac39-4d44-bf48-1cdc8cfb341c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:19.700490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27688835-ac39-4d44-bf48-1cdc8cfb341c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:19.700767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27688835-ac39-4d44-bf48-1cdc8cfb341c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:19.701604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27688835-ac39-4d44-bf48-1cdc8cfb341c None admin] RBAC: Adding query filter params (role.id=1399a838106b4f9bb5fa1c942cc3ef22, user.id=e02fde0d54cd4f09a8f614bcf20be44b, scope.system=all) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:19.701970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27688835-ac39-4d44-bf48-1cdc8cfb341c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:19.703430 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27688835-ac39-4d44-bf48-1cdc8cfb341c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:19.710670 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:19.720316 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 172/334] 10.222.0.94 () {56 vars in 1308 bytes} [Mon Jul 18 13:55:19 2022] GET /identity/v3/role_assignments?user.id=e02fde0d54cd4f09a8f614bcf20be44b&scope.system=all&role.id=1399a838106b4f9bb5fa1c942cc3ef22 => generated 510 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:21.437726 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b29d00-e3ab-43cb-8f12-7bbdcb1fb9ab None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:21.438245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b29d00-e3ab-43cb-8f12-7bbdcb1fb9ab None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:21.438556 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b29d00-e3ab-43cb-8f12-7bbdcb1fb9ab None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:21.440203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 163/335] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:21.453439 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522a550-9484-4ccb-899e-10a9808e57ba None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:21.454568 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522a550-9484-4ccb-899e-10a9808e57ba None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:21.455049 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522a550-9484-4ccb-899e-10a9808e57ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:21.476935 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e522a550-9484-4ccb-899e-10a9808e57ba None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:21.478985 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e522a550-9484-4ccb-899e-10a9808e57ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:21.566581 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 173/336] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:21.600242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b29d00-e3ab-43cb-8f12-7bbdcb1fb9ab None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:21.733179 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:21.757211 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b29d00-e3ab-43cb-8f12-7bbdcb1fb9ab None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:21.802502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d7b32-b12a-49fc-8c61-9ade718b5fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:21.803320 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d7b32-b12a-49fc-8c61-9ade718b5fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:21.803608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d7b32-b12a-49fc-8c61-9ade718b5fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:21.803886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d7b32-b12a-49fc-8c61-9ade718b5fc3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:21.821573 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d7b32-b12a-49fc-8c61-9ade718b5fc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:21.829568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-553d7b32-b12a-49fc-8c61-9ade718b5fc3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:21.829568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:21.829568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:21.829568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:21.829568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:21.829568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:21.829568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:21.829568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:21.829568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:21.829568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:21.829568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:21.829568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:21.829568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:21.829568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:21.829568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:21.829568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:21.830437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:21.830437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:21.830437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:21.830437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:21.830437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:21.830437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:21.830437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:21.830437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:21.830437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:21.830437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:21.830437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:21.830437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:21.830437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:21.830437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:21.830437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:21.830437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:21.831231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:21.831231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:21.831231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:21.831231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:21.831231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:21.831231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:21.831231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:21.831231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:21.831231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:21.831231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:21.831231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:21.831231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:21.839029 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 164/337] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:21.839167 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522a550-9484-4ccb-899e-10a9808e57ba None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:21.864150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383a914-ea7d-46ae-9d45-c160ddb78f6f None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:21.864150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383a914-ea7d-46ae-9d45-c160ddb78f6f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:21.864150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383a914-ea7d-46ae-9d45-c160ddb78f6f None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:21.865591 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 165/338] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:21.871386 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90dc33-5f4b-465e-b8ae-55b01e551940 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:21.871684 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90dc33-5f4b-465e-b8ae-55b01e551940 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:21.871978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90dc33-5f4b-465e-b8ae-55b01e551940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:21.907973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-7f90dc33-5f4b-465e-b8ae-55b01e551940 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:21.907973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7f90dc33-5f4b-465e-b8ae-55b01e551940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:21.997942 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522a550-9484-4ccb-899e-10a9808e57ba None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:22.028376 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:22.043916 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c5931d-aa0b-4355-8c32-1e2144168380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:22.044987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5931d-aa0b-4355-8c32-1e2144168380 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:22.045283 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5931d-aa0b-4355-8c32-1e2144168380 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:22.045666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5931d-aa0b-4355-8c32-1e2144168380 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:22.046429 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c5931d-aa0b-4355-8c32-1e2144168380 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:22.046742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c5931d-aa0b-4355-8c32-1e2144168380 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:22.048173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c5931d-aa0b-4355-8c32-1e2144168380 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:22.054558 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 166/339] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:22.064020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 174/340] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:55:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:22.069552 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90dc33-5f4b-465e-b8ae-55b01e551940 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:22.103783 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2355b9-a6ee-47a1-8305-37f55245912e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:22.104134 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2355b9-a6ee-47a1-8305-37f55245912e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:22.104422 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2355b9-a6ee-47a1-8305-37f55245912e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:22.106035 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 175/341] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:22.111136 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fd0cf-0f9f-40ef-be92-bf304d905086 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:22.111495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fd0cf-0f9f-40ef-be92-bf304d905086 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:22.111786 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fd0cf-0f9f-40ef-be92-bf304d905086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:22.141453 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4f7fd0cf-0f9f-40ef-be92-bf304d905086 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:22.142686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4f7fd0cf-0f9f-40ef-be92-bf304d905086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:22.171471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:22.221784 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90dc33-5f4b-465e-b8ae-55b01e551940 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:22.287383 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 176/342] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:22.291462 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7fd0cf-0f9f-40ef-be92-bf304d905086 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:22.292664 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1869f-e673-491d-8d53-53a777309dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:22.293565 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1869f-e673-491d-8d53-53a777309dab None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:22.294162 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1869f-e673-491d-8d53-53a777309dab None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:22.294728 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1869f-e673-491d-8d53-53a777309dab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:22.413732 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7fd0cf-0f9f-40ef-be92-bf304d905086 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:22.437367 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:22.443737 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1869f-e673-491d-8d53-53a777309dab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:22.446727 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1869f-e673-491d-8d53-53a777309dab None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:22.473046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe8453-37a4-446e-85ce-08e4b3734d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:22.474553 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe8453-37a4-446e-85ce-08e4b3734d30 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:22.476532 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe8453-37a4-446e-85ce-08e4b3734d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:22.477029 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe8453-37a4-446e-85ce-08e4b3734d30 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:22.499494 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe8453-37a4-446e-85ce-08e4b3734d30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:22.510050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e6fe8453-37a4-446e-85ce-08e4b3734d30 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:55:22.510050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:22.510050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:22.510050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:22.510050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:22.510050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:22.510050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:22.510050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:22.510050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:22.510050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:22.510050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:22.510050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:22.510050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:22.510050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:22.510050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:22.510050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:22.510949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:22.510949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:22.510949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:22.510949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:22.510949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:22.510949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:22.510949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:22.510949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:22.510949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:22.510949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:22.510949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:22.510949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:22.510949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:22.510949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:22.510949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:22.510949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:22.511750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:22.511750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:22.511750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:22.511750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:22.511750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:22.511750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:22.511750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:22.511750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:22.511750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:22.511750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:22.511750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:55:22.511750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:22.514342 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 177/343] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Jul 18 13:55:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:22.515303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe8453-37a4-446e-85ce-08e4b3734d30 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:22.561315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:22.598294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 167/344] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:22.599755 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeca573-e5ec-4bf0-aaaa-cdc6db3a14b4 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:22.600079 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeca573-e5ec-4bf0-aaaa-cdc6db3a14b4 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:22.600365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeca573-e5ec-4bf0-aaaa-cdc6db3a14b4 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:22.608282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 168/345] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:22.609124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeca573-e5ec-4bf0-aaaa-cdc6db3a14b4 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:22.634994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe8453-37a4-446e-85ce-08e4b3734d30 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:22.671158 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf0c3a-8ad5-49b8-9277-1d1f08303ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:22.673399 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf0c3a-8ad5-49b8-9277-1d1f08303ebd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:22.673924 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf0c3a-8ad5-49b8-9277-1d1f08303ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:22.674410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf0c3a-8ad5-49b8-9277-1d1f08303ebd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:22.687323 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bf0c3a-8ad5-49b8-9277-1d1f08303ebd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:22.695785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-72bf0c3a-8ad5-49b8-9277-1d1f08303ebd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:22.695785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:22.695785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:22.695785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:22.695785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:22.695785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:22.695785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:22.695785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:22.695785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:22.695785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:22.695785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:22.695785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:22.695785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:22.695785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:22.695785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:22.695785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:22.696854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:22.696854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:22.696854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:22.696854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:22.696854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:22.696854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:22.696854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:22.696854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:22.696854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:22.696854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:22.696854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:22.696854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:22.696854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:22.696854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:22.696854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:22.696854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:22.697674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:22.697674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:22.697674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:22.697674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:22.697674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:22.697674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:22.697674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:22.697674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:22.697674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:22.697674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:22.697674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:22.697674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:22.700632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 178/346] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:22.701774 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf0c3a-8ad5-49b8-9277-1d1f08303ebd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:22.733798 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeca573-e5ec-4bf0-aaaa-cdc6db3a14b4 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:22.764384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:22.774648 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2a94f-beb9-413b-b2c2-d4afa757c7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:22.775633 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2a94f-beb9-413b-b2c2-d4afa757c7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:22.775633 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2a94f-beb9-413b-b2c2-d4afa757c7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:22.775912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2a94f-beb9-413b-b2c2-d4afa757c7b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:22.776793 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2a94f-beb9-413b-b2c2-d4afa757c7b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:22.777005 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2a94f-beb9-413b-b2c2-d4afa757c7b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:22.779005 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2a94f-beb9-413b-b2c2-d4afa757c7b8 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:22.790306 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 169/347] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:22.796558 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe138de-1f3e-4056-93d2-d8572562832b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:22.796864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe138de-1f3e-4056-93d2-d8572562832b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:22.797154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe138de-1f3e-4056-93d2-d8572562832b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:22.818441 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf0c3a-8ad5-49b8-9277-1d1f08303ebd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:22.822262 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-dfe138de-1f3e-4056-93d2-d8572562832b None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:22.822978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-dfe138de-1f3e-4056-93d2-d8572562832b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:22.858123 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:22.861233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ece3a-d982-4712-a84f-1eaddbb4211c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:22.861651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ece3a-d982-4712-a84f-1eaddbb4211c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:22.862280 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ece3a-d982-4712-a84f-1eaddbb4211c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:22.862565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ece3a-d982-4712-a84f-1eaddbb4211c None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:22.863907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53ece3a-d982-4712-a84f-1eaddbb4211c None admin] RBAC: Authorizing `identity:get_project(project_id=ec42cd390d9e4d07bb7573020b5b1003)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:22.865951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53ece3a-d982-4712-a84f-1eaddbb4211c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:22.872911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 179/348] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:22 2022] GET /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:22.872911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ece3a-d982-4712-a84f-1eaddbb4211c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:22.923635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 170/349] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:22.925215 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe138de-1f3e-4056-93d2-d8572562832b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:23.011987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ece3a-d982-4712-a84f-1eaddbb4211c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:23.022758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:23.059444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10751fa7-44cf-4315-ac2c-dc117b63e0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:23.060455 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10751fa7-44cf-4315-ac2c-dc117b63e0e8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:23.060891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10751fa7-44cf-4315-ac2c-dc117b63e0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:23.061313 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10751fa7-44cf-4315-ac2c-dc117b63e0e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:23.061977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe138de-1f3e-4056-93d2-d8572562832b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:23.062382 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10751fa7-44cf-4315-ac2c-dc117b63e0e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:23.062837 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10751fa7-44cf-4315-ac2c-dc117b63e0e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:23.064645 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10751fa7-44cf-4315-ac2c-dc117b63e0e8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:23.087073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 180/350] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:55:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:23.089629 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10751fa7-44cf-4315-ac2c-dc117b63e0e8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:23.122829 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed667f-1781-4407-a221-6d048b413a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:23.123450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed667f-1781-4407-a221-6d048b413a0a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:23.123738 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed667f-1781-4407-a221-6d048b413a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:23.124174 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed667f-1781-4407-a221-6d048b413a0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:23.157746 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:23.199525 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed667f-1781-4407-a221-6d048b413a0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:23.201029 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed667f-1781-4407-a221-6d048b413a0a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:23.218418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10751fa7-44cf-4315-ac2c-dc117b63e0e8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:23.257449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374b4ad-3d9c-43e5-a976-b5a042b22cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:23.259309 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374b4ad-3d9c-43e5-a976-b5a042b22cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:23.259834 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374b4ad-3d9c-43e5-a976-b5a042b22cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:23.260328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374b4ad-3d9c-43e5-a976-b5a042b22cf4 None admin] PATH_INFO: `/v3/groups/946558619abc498998f6c559e8791687` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:23.267196 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2374b4ad-3d9c-43e5-a976-b5a042b22cf4 None admin] RBAC: Authorizing `identity:get_group(group_id=946558619abc498998f6c559e8791687)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:23.269023 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2374b4ad-3d9c-43e5-a976-b5a042b22cf4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:23.271735 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 181/351] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Jul 18 13:55:23 2022] GET /identity/v3/groups/946558619abc498998f6c559e8791687 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:23.272844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374b4ad-3d9c-43e5-a976-b5a042b22cf4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:23.313234 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:23.315813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 171/352] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:23.316532 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed667f-1781-4407-a221-6d048b413a0a None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:23.388424 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:23.389329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374b4ad-3d9c-43e5-a976-b5a042b22cf4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:23.425726 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed667f-1781-4407-a221-6d048b413a0a None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:23.428238 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65e093-ce56-4a2d-8d69-dba0b8492030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:23.429301 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65e093-ce56-4a2d-8d69-dba0b8492030 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:23.429790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65e093-ce56-4a2d-8d69-dba0b8492030 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:23.430269 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65e093-ce56-4a2d-8d69-dba0b8492030 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:23.444872 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c65e093-ce56-4a2d-8d69-dba0b8492030 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:23.453142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5c65e093-ce56-4a2d-8d69-dba0b8492030 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 13:55:23.453142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:23.453142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:23.453142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:23.453142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:23.453142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:23.453142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:23.453142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:23.453142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:23.453142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:23.453142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:23.453142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:23.453142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:23.453142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:23.453142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:23.453142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:23.454304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:23.454304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:23.454304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:23.454304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:23.454304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:23.454304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:23.454304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:23.454304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:23.454304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:23.454304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:23.454304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:23.454304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:23.454304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:23.454304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:23.454304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:23.454304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:23.455096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:23.455096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:23.455096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:23.455096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:23.455096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:23.455096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:23.455096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:23.455096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:23.455096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:23.455096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:23.455096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 13:55:23.455096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:23.455692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 182/353] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:23.462678 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65e093-ce56-4a2d-8d69-dba0b8492030 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:23.464715 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfc959-74b6-49cb-9c79-ccab5dc22d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:23.465153 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfc959-74b6-49cb-9c79-ccab5dc22d72 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:23.465364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfc959-74b6-49cb-9c79-ccab5dc22d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:23.465642 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfc959-74b6-49cb-9c79-ccab5dc22d72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:23.557728 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfc959-74b6-49cb-9c79-ccab5dc22d72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:23.559825 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfc959-74b6-49cb-9c79-ccab5dc22d72 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:23.578282 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:23.586270 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65e093-ce56-4a2d-8d69-dba0b8492030 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:23.624197 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c37e9-6d80-4c04-bf31-5e25a6046250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:23.624865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c37e9-6d80-4c04-bf31-5e25a6046250 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:23.625256 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c37e9-6d80-4c04-bf31-5e25a6046250 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:23.625544 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c37e9-6d80-4c04-bf31-5e25a6046250 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:23.626394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4c37e9-6d80-4c04-bf31-5e25a6046250 None admin] RBAC: Adding query filter params (group.id=946558619abc498998f6c559e8791687, role.id=1399a838106b4f9bb5fa1c942cc3ef22, scope.project.id=ec42cd390d9e4d07bb7573020b5b1003) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:23.626824 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4c37e9-6d80-4c04-bf31-5e25a6046250 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:23.628339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4c37e9-6d80-4c04-bf31-5e25a6046250 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:23.634990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 183/354] 10.222.0.94 () {56 vars in 1376 bytes} [Mon Jul 18 13:55:23 2022] GET /identity/v3/role_assignments?group.id=946558619abc498998f6c559e8791687&scope.project.id=ec42cd390d9e4d07bb7573020b5b1003&role.id=1399a838106b4f9bb5fa1c942cc3ef22 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:23.635710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c37e9-6d80-4c04-bf31-5e25a6046250 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:23.645575 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:23.700272 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 172/355] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:23.702448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527e945-e561-45af-bff7-2dc2cd6e5ed6 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:23.702921 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527e945-e561-45af-bff7-2dc2cd6e5ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:23.703390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527e945-e561-45af-bff7-2dc2cd6e5ed6 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:23.705381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 173/356] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:23.706517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527e945-e561-45af-bff7-2dc2cd6e5ed6 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:23.767484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c37e9-6d80-4c04-bf31-5e25a6046250 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:23.809874 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52744bd7-335f-4389-be95-21cb3aec69d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:23.810770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52744bd7-335f-4389-be95-21cb3aec69d0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:23.811056 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52744bd7-335f-4389-be95-21cb3aec69d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:23.811334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52744bd7-335f-4389-be95-21cb3aec69d0 None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:23.812661 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52744bd7-335f-4389-be95-21cb3aec69d0 None admin] RBAC: Authorizing `identity:get_project(project_id=ec42cd390d9e4d07bb7573020b5b1003)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:23.814146 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52744bd7-335f-4389-be95-21cb3aec69d0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:23.816808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 184/357] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:23 2022] GET /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:23.817198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52744bd7-335f-4389-be95-21cb3aec69d0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:23.848566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527e945-e561-45af-bff7-2dc2cd6e5ed6 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:23.859500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:23.900865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a4b63-9a70-4f7a-a704-d1119f64d9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:23.901539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a4b63-9a70-4f7a-a704-d1119f64d9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:23.901824 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a4b63-9a70-4f7a-a704-d1119f64d9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:23.902100 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a4b63-9a70-4f7a-a704-d1119f64d9c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:23.902854 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a4b63-9a70-4f7a-a704-d1119f64d9c1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:23.903284 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a4b63-9a70-4f7a-a704-d1119f64d9c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:23.904848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a4b63-9a70-4f7a-a704-d1119f64d9c1 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:23.913938 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 174/358] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:55:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:23.915291 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61c948-80bc-41a2-b1e4-0a3c5df59b3f None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:23.915581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61c948-80bc-41a2-b1e4-0a3c5df59b3f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:23.915865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61c948-80bc-41a2-b1e4-0a3c5df59b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:23.932779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-0d61c948-80bc-41a2-b1e4-0a3c5df59b3f None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:23.934865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0d61c948-80bc-41a2-b1e4-0a3c5df59b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:23.950920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52744bd7-335f-4389-be95-21cb3aec69d0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:23.968126 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:23.995709 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4f292-e068-4b01-8c82-7ca2ecda59d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:23.996646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4f292-e068-4b01-8c82-7ca2ecda59d0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:23.996938 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4f292-e068-4b01-8c82-7ca2ecda59d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:23.997226 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4f292-e068-4b01-8c82-7ca2ecda59d0 None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:23.998562 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc4f292-e068-4b01-8c82-7ca2ecda59d0 None admin] RBAC: Authorizing `identity:get_project(project_id=7bfe28b4f0f648dd906926f955192736)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:24.000164 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc4f292-e068-4b01-8c82-7ca2ecda59d0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:24.001880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 185/359] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:23 2022] GET /identity/v3/projects/7bfe28b4f0f648dd906926f955192736 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:24.003071 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247953e0-75a0-4bb9-8336-ebd396005b97 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:24.003364 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247953e0-75a0-4bb9-8336-ebd396005b97 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:24.003646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247953e0-75a0-4bb9-8336-ebd396005b97 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:24.005166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 186/360] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:24.005875 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247953e0-75a0-4bb9-8336-ebd396005b97 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:24.045923 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 175/361] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:24.046698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61c948-80bc-41a2-b1e4-0a3c5df59b3f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:24.134120 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:24.144256 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247953e0-75a0-4bb9-8336-ebd396005b97 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:24.175430 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61c948-80bc-41a2-b1e4-0a3c5df59b3f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:24.189856 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25980a-eee7-4728-8512-f7e509321879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:24.190725 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25980a-eee7-4728-8512-f7e509321879 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:24.191106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25980a-eee7-4728-8512-f7e509321879 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:24.191390 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25980a-eee7-4728-8512-f7e509321879 None admin] PATH_INFO: `/v3/groups/946558619abc498998f6c559e8791687` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:24.192917 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25980a-eee7-4728-8512-f7e509321879 None admin] RBAC: Authorizing `identity:get_group(group_id=946558619abc498998f6c559e8791687)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:24.194522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25980a-eee7-4728-8512-f7e509321879 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:24.196514 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 187/362] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Jul 18 13:55:24 2022] GET /identity/v3/groups/946558619abc498998f6c559e8791687 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:24.197594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82e9ac-4d4a-4188-8cba-2b2c324c8a95 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:24.197890 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82e9ac-4d4a-4188-8cba-2b2c324c8a95 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:24.198169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82e9ac-4d4a-4188-8cba-2b2c324c8a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:24.221435 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6d82e9ac-4d4a-4188-8cba-2b2c324c8a95 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:24.221732 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e2aeb-59e9-4c2a-a24b-085723888bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:24.222638 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6d82e9ac-4d4a-4188-8cba-2b2c324c8a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:24.222742 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e2aeb-59e9-4c2a-a24b-085723888bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:24.222923 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e2aeb-59e9-4c2a-a24b-085723888bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:24.223199 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e2aeb-59e9-4c2a-a24b-085723888bb7 None admin] PATH_INFO: `/v3/users/ad57f4b94892434db8c7b1e1d491843a` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:24.225232 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e2aeb-59e9-4c2a-a24b-085723888bb7 None admin] RBAC: Authorizing `identity:get_user(user_id=ad57f4b94892434db8c7b1e1d491843a)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:24.226010 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e2aeb-59e9-4c2a-a24b-085723888bb7 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:24.228550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 176/363] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:55:24 2022] GET /identity/v3/users/ad57f4b94892434db8c7b1e1d491843a => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:24.229136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e2aeb-59e9-4c2a-a24b-085723888bb7 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:24.271237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:24.325799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 188/364] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:24.327991 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82e9ac-4d4a-4188-8cba-2b2c324c8a95 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:24.385943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e2aeb-59e9-4c2a-a24b-085723888bb7 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:24.437231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1734a4-352a-4b84-bd9d-31517b7be70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:24.438025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1734a4-352a-4b84-bd9d-31517b7be70d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:24.438308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1734a4-352a-4b84-bd9d-31517b7be70d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:24.438431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:24.438589 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1734a4-352a-4b84-bd9d-31517b7be70d None admin] PATH_INFO: `/v3/groups/946558619abc498998f6c559e8791687` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:24.440754 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1734a4-352a-4b84-bd9d-31517b7be70d None admin] RBAC: Authorizing `identity:get_group(group_id=946558619abc498998f6c559e8791687)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:24.443906 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1734a4-352a-4b84-bd9d-31517b7be70d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:24.447556 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 177/365] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Jul 18 13:55:24 2022] GET /identity/v3/groups/946558619abc498998f6c559e8791687 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:24.451043 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1734a4-352a-4b84-bd9d-31517b7be70d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:24.454893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82e9ac-4d4a-4188-8cba-2b2c324c8a95 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:24.497542 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb193c-727b-4fe4-9745-85358220f1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:24.498750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb193c-727b-4fe4-9745-85358220f1aa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:24.499244 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb193c-727b-4fe4-9745-85358220f1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:24.499525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb193c-727b-4fe4-9745-85358220f1aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:24.512366 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb193c-727b-4fe4-9745-85358220f1aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:24.521465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b4bb193c-727b-4fe4-9745-85358220f1aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:55:24.521465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:24.521465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:24.521465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:24.521465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:24.521465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:24.521465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:24.521465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:24.521465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:24.521465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:24.521465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:24.521465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:24.521465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:24.521465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:24.521465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:24.521465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:24.522627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:24.522627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:24.522627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:24.522627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:24.522627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:24.522627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:24.522627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:24.522627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:24.522627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:24.522627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:24.522627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:24.522627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:24.522627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:24.522627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:24.522627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:24.522627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:24.523431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:24.523431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:24.523431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:24.523431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:24.523431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:24.523431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:24.523431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:24.523431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:24.523431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:24.523431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:24.523431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:55:24.523431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:24.523431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 189/366] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Jul 18 13:55:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:24.524198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb193c-727b-4fe4-9745-85358220f1aa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:24.575797 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:24.589171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1734a4-352a-4b84-bd9d-31517b7be70d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:24.632577 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865eaf1-d9f2-4862-b933-e5c922cce97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:24.632959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865eaf1-d9f2-4862-b933-e5c922cce97c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:24.633246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865eaf1-d9f2-4862-b933-e5c922cce97c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:24.633558 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865eaf1-d9f2-4862-b933-e5c922cce97c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:24.634392 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865eaf1-d9f2-4862-b933-e5c922cce97c None admin] RBAC: Adding query filter params (group.id=946558619abc498998f6c559e8791687, role.id=580aae575f414b3c9b4ccf1e27e0b640, scope.project.id=ec42cd390d9e4d07bb7573020b5b1003) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:24.634709 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865eaf1-d9f2-4862-b933-e5c922cce97c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:24.636197 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865eaf1-d9f2-4862-b933-e5c922cce97c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:24.642642 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 178/367] 10.222.0.94 () {56 vars in 1376 bytes} [Mon Jul 18 13:55:24 2022] GET /identity/v3/role_assignments?group.id=946558619abc498998f6c559e8791687&scope.project.id=ec42cd390d9e4d07bb7573020b5b1003&role.id=580aae575f414b3c9b4ccf1e27e0b640 => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:24.643983 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865eaf1-d9f2-4862-b933-e5c922cce97c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:24.648309 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb193c-727b-4fe4-9745-85358220f1aa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:24.702034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca287b5-968b-4bb0-8d3a-a205aa6096c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:24.702649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca287b5-968b-4bb0-8d3a-a205aa6096c8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:24.703214 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca287b5-968b-4bb0-8d3a-a205aa6096c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:24.703591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca287b5-968b-4bb0-8d3a-a205aa6096c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:24.705333 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca287b5-968b-4bb0-8d3a-a205aa6096c8 None admin] RBAC: Adding query filter params (role.id=fd276ad89f2d4192b4f90e126d9362d3, user.id=ad57f4b94892434db8c7b1e1d491843a, scope.system=all) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:24.705658 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca287b5-968b-4bb0-8d3a-a205aa6096c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:24.707179 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca287b5-968b-4bb0-8d3a-a205aa6096c8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:24.732135 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 190/368] 10.222.0.94 () {56 vars in 1308 bytes} [Mon Jul 18 13:55:24 2022] GET /identity/v3/role_assignments?user.id=ad57f4b94892434db8c7b1e1d491843a&scope.system=all&role.id=fd276ad89f2d4192b4f90e126d9362d3 => generated 510 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:24.733183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca287b5-968b-4bb0-8d3a-a205aa6096c8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:24.758293 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:24.778049 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865eaf1-d9f2-4862-b933-e5c922cce97c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:24.825327 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a484f05-d7b9-4e99-81f9-922893125522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:24.826146 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a484f05-d7b9-4e99-81f9-922893125522 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:24.826441 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a484f05-d7b9-4e99-81f9-922893125522 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:24.826722 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a484f05-d7b9-4e99-81f9-922893125522 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:24.841836 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a484f05-d7b9-4e99-81f9-922893125522 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:24.849853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-3a484f05-d7b9-4e99-81f9-922893125522 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:55:24.849853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:24.849853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:24.849853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:24.849853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:24.849853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:24.849853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:24.849853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:24.849853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:24.849853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:24.849853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:24.849853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:24.849853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:24.849853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:24.849853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:24.849853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:24.850815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:24.850815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:24.850815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:24.850815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:24.850815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:24.850815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:24.850815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:24.850815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:24.850815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:24.850815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:24.850815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:24.850815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:24.850815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:24.850815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:24.850815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:24.850815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:24.851596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:24.851596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:24.851596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:24.851596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:24.851596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:24.851596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:24.851596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:24.851596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:24.851596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:24.851596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:24.851596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:55:24.851596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:24.851596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 179/369] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:55:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:24.853741 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a484f05-d7b9-4e99-81f9-922893125522 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:24.880258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:24.972059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca287b5-968b-4bb0-8d3a-a205aa6096c8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:24.994997 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a484f05-d7b9-4e99-81f9-922893125522 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:25.018356 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58a014-42f2-48c3-919d-4b0b0d88d54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:25.019037 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58a014-42f2-48c3-919d-4b0b0d88d54e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:25.019635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58a014-42f2-48c3-919d-4b0b0d88d54e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:25.019928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f58a014-42f2-48c3-919d-4b0b0d88d54e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:25.020805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f58a014-42f2-48c3-919d-4b0b0d88d54e None admin] RBAC: Adding query filter params (group.id=946558619abc498998f6c559e8791687, role.id=1399a838106b4f9bb5fa1c942cc3ef22, scope.project.id=7bfe28b4f0f648dd906926f955192736) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:25.021131 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f58a014-42f2-48c3-919d-4b0b0d88d54e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:25.023069 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f58a014-42f2-48c3-919d-4b0b0d88d54e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:25.032990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 191/370] 10.222.0.94 () {56 vars in 1376 bytes} [Mon Jul 18 13:55:24 2022] GET /identity/v3/role_assignments?group.id=946558619abc498998f6c559e8791687&scope.project.id=7bfe28b4f0f648dd906926f955192736&role.id=1399a838106b4f9bb5fa1c942cc3ef22 => generated 260 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:25.033909 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58a014-42f2-48c3-919d-4b0b0d88d54e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:25.041720 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428eebe7-0212-49d4-9f53-432cb23e7d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:25.042416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428eebe7-0212-49d4-9f53-432cb23e7d19 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:25.042707 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428eebe7-0212-49d4-9f53-432cb23e7d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:25.042994 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428eebe7-0212-49d4-9f53-432cb23e7d19 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:25.043759 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428eebe7-0212-49d4-9f53-432cb23e7d19 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:25.044144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428eebe7-0212-49d4-9f53-432cb23e7d19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:25.045577 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428eebe7-0212-49d4-9f53-432cb23e7d19 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:25.054874 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 180/371] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:25.060678 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428eebe7-0212-49d4-9f53-432cb23e7d19 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:25.159714 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:25.192321 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f58a014-42f2-48c3-919d-4b0b0d88d54e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:25.222226 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:25.241012 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73a9d9-5ad5-4337-bad6-9da75f607fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:25.241720 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73a9d9-5ad5-4337-bad6-9da75f607fce None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:25.242010 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73a9d9-5ad5-4337-bad6-9da75f607fce None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:25.242292 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73a9d9-5ad5-4337-bad6-9da75f607fce None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:25.243738 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73a9d9-5ad5-4337-bad6-9da75f607fce None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:25.244087 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73a9d9-5ad5-4337-bad6-9da75f607fce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:25.245785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73a9d9-5ad5-4337-bad6-9da75f607fce None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:25.264811 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 192/372] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:25.267206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428eebe7-0212-49d4-9f53-432cb23e7d19 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:25.269738 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73a9d9-5ad5-4337-bad6-9da75f607fce None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:25.338914 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2bdce1-e8fd-4462-8173-c4ff9ebfa83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:25.339586 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2bdce1-e8fd-4462-8173-c4ff9ebfa83e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:25.339877 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2bdce1-e8fd-4462-8173-c4ff9ebfa83e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:25.340184 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2bdce1-e8fd-4462-8173-c4ff9ebfa83e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:25.415987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73a9d9-5ad5-4337-bad6-9da75f607fce None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:25.457015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2bdce1-e8fd-4462-8173-c4ff9ebfa83e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:25.458477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2bdce1-e8fd-4462-8173-c4ff9ebfa83e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:25.478183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f14f5-ba02-422b-9cf6-939891eac127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:25.478750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f14f5-ba02-422b-9cf6-939891eac127 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:25.479074 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f14f5-ba02-422b-9cf6-939891eac127 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:25.481207 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f14f5-ba02-422b-9cf6-939891eac127 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:25.489381 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:25.595808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:25.612511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f14f5-ba02-422b-9cf6-939891eac127 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:25.616232 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f14f5-ba02-422b-9cf6-939891eac127 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:25.642953 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 181/373] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 13:55:25.649867 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2bdce1-e8fd-4462-8173-c4ff9ebfa83e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:25.799066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 193/374] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:25.804800 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f14f5-ba02-422b-9cf6-939891eac127 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:25.823644 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2bdce1-e8fd-4462-8173-c4ff9ebfa83e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:25.858068 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:25.902631 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b83e4b-f064-40da-82cf-d7901f4a516e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:25.903560 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b83e4b-f064-40da-82cf-d7901f4a516e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:25.903855 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b83e4b-f064-40da-82cf-d7901f4a516e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:25.904227 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b83e4b-f064-40da-82cf-d7901f4a516e None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:25.908683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b83e4b-f064-40da-82cf-d7901f4a516e None admin] RBAC: Authorizing `identity:get_project(project_id=7bfe28b4f0f648dd906926f955192736)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:25.910178 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b83e4b-f064-40da-82cf-d7901f4a516e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:25.912282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 182/375] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:25 2022] GET /identity/v3/projects/7bfe28b4f0f648dd906926f955192736 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:25.917946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b83e4b-f064-40da-82cf-d7901f4a516e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:25.938095 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f14f5-ba02-422b-9cf6-939891eac127 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:25.976315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15da463-0c74-4b95-b308-28f9ffb11964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:25.977728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15da463-0c74-4b95-b308-28f9ffb11964 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:25.978021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15da463-0c74-4b95-b308-28f9ffb11964 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:25.978303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15da463-0c74-4b95-b308-28f9ffb11964 None admin] PATH_INFO: `/v3/projects/defeadff0f9c417facade99f9905fca6` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:25.979545 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15da463-0c74-4b95-b308-28f9ffb11964 None admin] RBAC: Authorizing `identity:get_project(project_id=defeadff0f9c417facade99f9905fca6)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:25.981042 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15da463-0c74-4b95-b308-28f9ffb11964 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:25.983073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 194/376] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:25 2022] GET /identity/v3/projects/defeadff0f9c417facade99f9905fca6 => generated 344 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:25.984369 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a39e5a-6725-46f0-9bae-8bf13d7df000 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:25.984663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a39e5a-6725-46f0-9bae-8bf13d7df000 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:25.984947 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a39e5a-6725-46f0-9bae-8bf13d7df000 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:25.986433 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 195/377] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:25.990478 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a39e5a-6725-46f0-9bae-8bf13d7df000 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:26.008248 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:26.068798 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b83e4b-f064-40da-82cf-d7901f4a516e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:26.103824 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a39e5a-6725-46f0-9bae-8bf13d7df000 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:26.105656 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d6f036-4ec0-44bf-805e-bc4e07ece6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:26.106526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6f036-4ec0-44bf-805e-bc4e07ece6eb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:26.106631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:26.106807 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6f036-4ec0-44bf-805e-bc4e07ece6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:26.107090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6f036-4ec0-44bf-805e-bc4e07ece6eb None admin] PATH_INFO: `/v3/groups/946558619abc498998f6c559e8791687` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:26.110162 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d6f036-4ec0-44bf-805e-bc4e07ece6eb None admin] RBAC: Authorizing `identity:get_group(group_id=946558619abc498998f6c559e8791687)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:26.111511 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d6f036-4ec0-44bf-805e-bc4e07ece6eb None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:26.113513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 183/378] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Jul 18 13:55:25 2022] GET /identity/v3/groups/946558619abc498998f6c559e8791687 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:26.114891 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1e929-0867-4299-b42b-d673a3ef0f62 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:26.115186 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1e929-0867-4299-b42b-d673a3ef0f62 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:26.115469 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1e929-0867-4299-b42b-d673a3ef0f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:26.133422 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-27c1e929-0867-4299-b42b-d673a3ef0f62 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:26.135133 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-27c1e929-0867-4299-b42b-d673a3ef0f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:26.149786 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5675e-e7bf-4ab1-b191-865fa32f46d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:26.150670 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5675e-e7bf-4ab1-b191-865fa32f46d8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:26.152091 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5675e-e7bf-4ab1-b191-865fa32f46d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:26.152437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c5675e-e7bf-4ab1-b191-865fa32f46d8 None admin] PATH_INFO: `/v3/groups/e1306e19450043b4939c3d57f7452e3c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:26.158968 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c5675e-e7bf-4ab1-b191-865fa32f46d8 None admin] RBAC: Authorizing `identity:get_group(group_id=e1306e19450043b4939c3d57f7452e3c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:26.161030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c5675e-e7bf-4ab1-b191-865fa32f46d8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:26.163406 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 196/379] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Jul 18 13:55:25 2022] GET /identity/v3/groups/e1306e19450043b4939c3d57f7452e3c => generated 229 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:26.164403 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5675e-e7bf-4ab1-b191-865fa32f46d8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:26.223971 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 184/380] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:26.224992 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1e929-0867-4299-b42b-d673a3ef0f62 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:26.287864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:26.295104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c5675e-e7bf-4ab1-b191-865fa32f46d8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:26.335178 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26259905-5ecb-4b68-8d7a-fac2d61af6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:26.336095 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26259905-5ecb-4b68-8d7a-fac2d61af6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:26.336603 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26259905-5ecb-4b68-8d7a-fac2d61af6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:26.337097 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26259905-5ecb-4b68-8d7a-fac2d61af6d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:26.338255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26259905-5ecb-4b68-8d7a-fac2d61af6d1 None admin] RBAC: Adding query filter params (group.id=946558619abc498998f6c559e8791687, role.id=580aae575f414b3c9b4ccf1e27e0b640, scope.project.id=7bfe28b4f0f648dd906926f955192736) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:26.338715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26259905-5ecb-4b68-8d7a-fac2d61af6d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:26.341116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26259905-5ecb-4b68-8d7a-fac2d61af6d1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:26.349433 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 197/381] 10.222.0.94 () {56 vars in 1376 bytes} [Mon Jul 18 13:55:26 2022] GET /identity/v3/role_assignments?group.id=946558619abc498998f6c559e8791687&scope.project.id=7bfe28b4f0f648dd906926f955192736&role.id=580aae575f414b3c9b4ccf1e27e0b640 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:26.350873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26259905-5ecb-4b68-8d7a-fac2d61af6d1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:26.353544 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1e929-0867-4299-b42b-d673a3ef0f62 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:26.394243 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:26.403784 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4082112-9e48-48ec-81b8-9dea0d2646ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:26.405385 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4082112-9e48-48ec-81b8-9dea0d2646ee None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:26.405914 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4082112-9e48-48ec-81b8-9dea0d2646ee None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:26.406396 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4082112-9e48-48ec-81b8-9dea0d2646ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:26.407836 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4082112-9e48-48ec-81b8-9dea0d2646ee None admin] RBAC: Adding query filter params (group.id=e1306e19450043b4939c3d57f7452e3c, role.id=c60a99de086f45c6820f7872a6b43feb, scope.project.id=defeadff0f9c417facade99f9905fca6) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:26.408737 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4082112-9e48-48ec-81b8-9dea0d2646ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:26.410867 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4082112-9e48-48ec-81b8-9dea0d2646ee None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:26.419222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 185/382] 10.222.0.94 () {56 vars in 1376 bytes} [Mon Jul 18 13:55:26 2022] GET /identity/v3/role_assignments?group.id=e1306e19450043b4939c3d57f7452e3c&scope.project.id=defeadff0f9c417facade99f9905fca6&role.id=c60a99de086f45c6820f7872a6b43feb => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:26.479673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26259905-5ecb-4b68-8d7a-fac2d61af6d1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:26.521031 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d69f0f-a096-4825-bc99-4ba98032575b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:26.522223 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d69f0f-a096-4825-bc99-4ba98032575b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:26.522720 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d69f0f-a096-4825-bc99-4ba98032575b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:26.523236 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d69f0f-a096-4825-bc99-4ba98032575b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:26.537167 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d69f0f-a096-4825-bc99-4ba98032575b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:26.545525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e3d69f0f-a096-4825-bc99-4ba98032575b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:26.545525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:26.545525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:26.545525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:26.545525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:26.545525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:26.545525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:26.545525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:26.545525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:26.545525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:26.545525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:26.545525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:26.545525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:26.545525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:26.545525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:26.545525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:26.546498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:26.546498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:26.546498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:26.546498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:26.546498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:26.546498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:26.546498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:26.546498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:26.546498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:26.546498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:26.546498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:26.546498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:26.546498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:26.546498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:26.546498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:26.546498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:26.547286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:26.547286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:26.547286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:26.547286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:26.547286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:26.547286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:26.547286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:26.547286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:26.547286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:26.547286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:26.547286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:26.547286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:26.549768 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 198/383] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:26.557112 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4082112-9e48-48ec-81b8-9dea0d2646ee None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:26.686059 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4082112-9e48-48ec-81b8-9dea0d2646ee None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:26.696464 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:26.729341 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0cf1e2-e82c-40d5-9526-785ff91751d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:26.730029 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0cf1e2-e82c-40d5-9526-785ff91751d3 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:26.730325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0cf1e2-e82c-40d5-9526-785ff91751d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:26.730606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0cf1e2-e82c-40d5-9526-785ff91751d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:26.731360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0cf1e2-e82c-40d5-9526-785ff91751d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:26.731671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0cf1e2-e82c-40d5-9526-785ff91751d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:26.733080 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0cf1e2-e82c-40d5-9526-785ff91751d3 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:26.742564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 186/384] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:55:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:26.757094 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d69f0f-a096-4825-bc99-4ba98032575b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:26.888021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:26.901076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d69f0f-a096-4825-bc99-4ba98032575b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:26.941866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee09e8-df07-479d-b078-ba29b778b885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:26.942677 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee09e8-df07-479d-b078-ba29b778b885 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:26.942967 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee09e8-df07-479d-b078-ba29b778b885 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:26.943252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee09e8-df07-479d-b078-ba29b778b885 None admin] PATH_INFO: `/v3/groups/946558619abc498998f6c559e8791687` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:26.946939 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee09e8-df07-479d-b078-ba29b778b885 None admin] RBAC: Authorizing `identity:get_group(group_id=946558619abc498998f6c559e8791687)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:26.948757 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee09e8-df07-479d-b078-ba29b778b885 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:26.951977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 199/385] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Jul 18 13:55:26 2022] GET /identity/v3/groups/946558619abc498998f6c559e8791687 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:26.957689 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0cf1e2-e82c-40d5-9526-785ff91751d3 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:27.089595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0cf1e2-e82c-40d5-9526-785ff91751d3 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:27.125615 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb36d0-eb5c-4751-8b5e-f90332c1e994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:27.126491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb36d0-eb5c-4751-8b5e-f90332c1e994 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:27.126777 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb36d0-eb5c-4751-8b5e-f90332c1e994 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:27.127054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb36d0-eb5c-4751-8b5e-f90332c1e994 None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:27.128266 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb36d0-eb5c-4751-8b5e-f90332c1e994 None admin] RBAC: Authorizing `identity:get_project(project_id=ec42cd390d9e4d07bb7573020b5b1003)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:27.129724 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb36d0-eb5c-4751-8b5e-f90332c1e994 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:27.131409 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 187/386] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:26 2022] GET /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:27.137506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee09e8-df07-479d-b078-ba29b778b885 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:27.287702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee09e8-df07-479d-b078-ba29b778b885 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:27.330577 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d382b3-e644-4647-ab23-d90a4546616e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:27.331481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d382b3-e644-4647-ab23-d90a4546616e None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:27.331771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d382b3-e644-4647-ab23-d90a4546616e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:27.332078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d382b3-e644-4647-ab23-d90a4546616e None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003/groups/946558619abc498998f6c559e8791687/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:27.334147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d382b3-e644-4647-ab23-d90a4546616e None admin] RBAC: Authorizing `identity:create_grant(project_id=ec42cd390d9e4d07bb7573020b5b1003, group_id=946558619abc498998f6c559e8791687, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:27.335662 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d382b3-e644-4647-ab23-d90a4546616e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:27.362998 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 200/387] 10.222.0.94 () {58 vars in 1442 bytes} [Mon Jul 18 13:55:27 2022] PUT /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003/groups/946558619abc498998f6c559e8791687/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:55:27.474312 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bb707-ea2f-4338-bb8f-98c4b24af369 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:27.474977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bb707-ea2f-4338-bb8f-98c4b24af369 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:27.475462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bb707-ea2f-4338-bb8f-98c4b24af369 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:27.477889 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 188/388] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:27.484157 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08b959-70b9-41e0-9562-3da76d72fe8c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:27.484461 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08b959-70b9-41e0-9562-3da76d72fe8c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:27.484747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08b959-70b9-41e0-9562-3da76d72fe8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:27.508855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ec08b959-70b9-41e0-9562-3da76d72fe8c None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:27.510121 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ec08b959-70b9-41e0-9562-3da76d72fe8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:27.532347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:27.600140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 201/389] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:27.638334 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07bb707-ea2f-4338-bb8f-98c4b24af369 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:27.686677 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:27.766849 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07bb707-ea2f-4338-bb8f-98c4b24af369 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:27.787351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4658036-a264-4255-a17e-77e3a7a32ae9 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:27.787704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4658036-a264-4255-a17e-77e3a7a32ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:27.788011 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4658036-a264-4255-a17e-77e3a7a32ae9 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:27.789393 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 202/390] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:27.794504 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72e82f-84e3-457c-9755-d45f36b5c281 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:27.794798 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72e82f-84e3-457c-9755-d45f36b5c281 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:27.795080 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72e82f-84e3-457c-9755-d45f36b5c281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:27.818599 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec85e9-00bd-4fd5-a256-cf2cd92f8fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:27.819797 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec85e9-00bd-4fd5-a256-cf2cd92f8fcb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:27.820315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec85e9-00bd-4fd5-a256-cf2cd92f8fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:27.820809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec85e9-00bd-4fd5-a256-cf2cd92f8fcb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:27.823985 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1e72e82f-84e3-457c-9755-d45f36b5c281 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:27.823985 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1e72e82f-84e3-457c-9755-d45f36b5c281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:27.849896 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec85e9-00bd-4fd5-a256-cf2cd92f8fcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:27.864635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-b5ec85e9-00bd-4fd5-a256-cf2cd92f8fcb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:55:27.864635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:27.864635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:27.864635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:27.864635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:27.864635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:27.864635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:27.864635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:27.864635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:27.864635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:27.864635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:27.864635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:27.864635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:27.864635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:27.864635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:27.864635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:27.865547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:27.865547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:27.865547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:27.865547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:27.865547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:27.865547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:27.865547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:27.865547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:27.865547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:27.865547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:27.865547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:27.865547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:27.865547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:27.865547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:27.865547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:27.865547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:27.866343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:27.866343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:27.866343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:27.866343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:27.866343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:27.866343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:27.866343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:27.866343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:27.866343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:27.866343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:27.866343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:55:27.866343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:27.866343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 189/391] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Jul 18 13:55:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:27.876963 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec85e9-00bd-4fd5-a256-cf2cd92f8fcb None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:27.947769 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 203/392] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:27.971877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72e82f-84e3-457c-9755-d45f36b5c281 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:28.064319 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:28.069413 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec85e9-00bd-4fd5-a256-cf2cd92f8fcb None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:28.117316 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:28.135500 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db75032-8582-4daf-af96-557563530255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:28.136220 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db75032-8582-4daf-af96-557563530255 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:28.136507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db75032-8582-4daf-af96-557563530255 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:28.136948 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db75032-8582-4daf-af96-557563530255 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:28.137701 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db75032-8582-4daf-af96-557563530255 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:28.138008 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db75032-8582-4daf-af96-557563530255 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:28.139376 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db75032-8582-4daf-af96-557563530255 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:28.145257 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72e82f-84e3-457c-9755-d45f36b5c281 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:28.149038 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 190/393] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:28.157314 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db75032-8582-4daf-af96-557563530255 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:28.217205 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51f5bd-02fd-4ff5-8adc-f8ab33393910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:28.218014 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51f5bd-02fd-4ff5-8adc-f8ab33393910 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:28.218300 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51f5bd-02fd-4ff5-8adc-f8ab33393910 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:28.218581 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51f5bd-02fd-4ff5-8adc-f8ab33393910 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:28.237929 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51f5bd-02fd-4ff5-8adc-f8ab33393910 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:28.245864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6b51f5bd-02fd-4ff5-8adc-f8ab33393910 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:28.245864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:28.245864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:28.245864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:28.245864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:28.245864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:28.245864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:28.245864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:28.245864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:28.245864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:28.245864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:28.245864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:28.245864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:28.245864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:28.245864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:28.245864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:28.246777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:28.246777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:28.246777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:28.246777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:28.246777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:28.246777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:28.246777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:28.246777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:28.246777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:28.246777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:28.246777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:28.246777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:28.246777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:28.246777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:28.246777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:28.246777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:28.247621 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:28.247621 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:28.247621 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:28.247621 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:28.247621 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:28.247621 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:28.247621 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:28.247621 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:28.247621 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:28.247621 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:28.247621 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:28.247621 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:28.248306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 204/394] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:55:28.254462 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51f5bd-02fd-4ff5-8adc-f8ab33393910 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:28.323895 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db75032-8582-4daf-af96-557563530255 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:28.373531 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc08d9-75b2-4928-83d7-0e2555b28a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:28.374751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc08d9-75b2-4928-83d7-0e2555b28a7e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:28.375252 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc08d9-75b2-4928-83d7-0e2555b28a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:28.375718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc08d9-75b2-4928-83d7-0e2555b28a7e None admin] PATH_INFO: `/v3/groups/946558619abc498998f6c559e8791687` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:28.377132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc08d9-75b2-4928-83d7-0e2555b28a7e None admin] RBAC: Authorizing `identity:get_group(group_id=946558619abc498998f6c559e8791687)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:28.378789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc08d9-75b2-4928-83d7-0e2555b28a7e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:28.382073 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 191/395] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Jul 18 13:55:28 2022] GET /identity/v3/groups/946558619abc498998f6c559e8791687 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:28.387229 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc08d9-75b2-4928-83d7-0e2555b28a7e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:28.422223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:28.460099 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51f5bd-02fd-4ff5-8adc-f8ab33393910 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:28.500625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea6abc-22d4-4c5a-a2af-807d9c605120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:28.501302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea6abc-22d4-4c5a-a2af-807d9c605120 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:28.501587 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea6abc-22d4-4c5a-a2af-807d9c605120 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:28.501871 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea6abc-22d4-4c5a-a2af-807d9c605120 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:28.502627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea6abc-22d4-4c5a-a2af-807d9c605120 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:28.502940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea6abc-22d4-4c5a-a2af-807d9c605120 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:28.504359 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea6abc-22d4-4c5a-a2af-807d9c605120 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:28.520940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 205/396] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:55:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:28.528896 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea6abc-22d4-4c5a-a2af-807d9c605120 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:28.532069 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:28.557162 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc08d9-75b2-4928-83d7-0e2555b28a7e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:28.592937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22642301-9549-4279-af73-08382e53b4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:28.597995 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22642301-9549-4279-af73-08382e53b4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:28.598447 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22642301-9549-4279-af73-08382e53b4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:28.598734 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22642301-9549-4279-af73-08382e53b4e5 None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:28.601750 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22642301-9549-4279-af73-08382e53b4e5 None admin] RBAC: Authorizing `identity:get_project(project_id=ec42cd390d9e4d07bb7573020b5b1003)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:28.603290 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22642301-9549-4279-af73-08382e53b4e5 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:28.607060 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 192/397] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:28 2022] GET /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:28.613132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22642301-9549-4279-af73-08382e53b4e5 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:28.705754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea6abc-22d4-4c5a-a2af-807d9c605120 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:28.773766 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66529f1-5756-42f4-8aa8-fee40484edfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:28.775192 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66529f1-5756-42f4-8aa8-fee40484edfd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:28.775479 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66529f1-5756-42f4-8aa8-fee40484edfd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:28.775761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66529f1-5756-42f4-8aa8-fee40484edfd None admin] PATH_INFO: `/v3/groups/946558619abc498998f6c559e8791687` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:28.778429 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66529f1-5756-42f4-8aa8-fee40484edfd None admin] RBAC: Authorizing `identity:get_group(group_id=946558619abc498998f6c559e8791687)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:28.779889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66529f1-5756-42f4-8aa8-fee40484edfd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:28.781745 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 206/398] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Jul 18 13:55:28 2022] GET /identity/v3/groups/946558619abc498998f6c559e8791687 => generated 226 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:28.786813 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66529f1-5756-42f4-8aa8-fee40484edfd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:28.796328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:28.800923 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22642301-9549-4279-af73-08382e53b4e5 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:28.871228 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d887f80-e921-4549-95fc-ca7282c2419d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:28.872147 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d887f80-e921-4549-95fc-ca7282c2419d None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:28.872439 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d887f80-e921-4549-95fc-ca7282c2419d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:28.872722 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d887f80-e921-4549-95fc-ca7282c2419d None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003/groups/946558619abc498998f6c559e8791687/roles/580aae575f414b3c9b4ccf1e27e0b640` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:28.877195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d887f80-e921-4549-95fc-ca7282c2419d None admin] RBAC: Authorizing `identity:create_grant(project_id=ec42cd390d9e4d07bb7573020b5b1003, group_id=946558619abc498998f6c559e8791687, role_id=580aae575f414b3c9b4ccf1e27e0b640)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:28.878666 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d887f80-e921-4549-95fc-ca7282c2419d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:28.902492 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 193/399] 10.222.0.94 () {58 vars in 1442 bytes} [Mon Jul 18 13:55:28 2022] PUT /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003/groups/946558619abc498998f6c559e8791687/roles/580aae575f414b3c9b4ccf1e27e0b640 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:55:28.912334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:28.938678 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b779c2-a67b-4411-9681-6cc6d42049d3 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:28.939018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b779c2-a67b-4411-9681-6cc6d42049d3 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:28.939258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b779c2-a67b-4411-9681-6cc6d42049d3 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:28.941373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 194/400] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:28.949337 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b43315-22a1-4395-87a8-2c6adcec7d49 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:28.949887 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b43315-22a1-4395-87a8-2c6adcec7d49 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:28.950324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b43315-22a1-4395-87a8-2c6adcec7d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:28.969808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66529f1-5756-42f4-8aa8-fee40484edfd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:28.984132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-73b43315-22a1-4395-87a8-2c6adcec7d49 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:28.985420 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-73b43315-22a1-4395-87a8-2c6adcec7d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:29.046178 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf1bd6-acb6-40be-b5e8-66eca13c08a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:29.047081 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf1bd6-acb6-40be-b5e8-66eca13c08a5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:29.047376 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf1bd6-acb6-40be-b5e8-66eca13c08a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:29.047658 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf1bd6-acb6-40be-b5e8-66eca13c08a5 None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:29.052750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf1bd6-acb6-40be-b5e8-66eca13c08a5 None admin] RBAC: Authorizing `identity:get_project(project_id=7bfe28b4f0f648dd906926f955192736)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:29.054257 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf1bd6-acb6-40be-b5e8-66eca13c08a5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:29.057046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 207/401] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:28 2022] GET /identity/v3/projects/7bfe28b4f0f648dd906926f955192736 => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:29.062206 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf1bd6-acb6-40be-b5e8-66eca13c08a5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:29.111461 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 195/402] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 13:55:29.133170 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b43315-22a1-4395-87a8-2c6adcec7d49 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:29.191026 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf1bd6-acb6-40be-b5e8-66eca13c08a5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:29.227230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcf85f-9f87-4233-a569-8263087df5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:29.228475 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcf85f-9f87-4233-a569-8263087df5df None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:29.228904 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcf85f-9f87-4233-a569-8263087df5df None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:29.229341 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcf85f-9f87-4233-a569-8263087df5df None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736/groups/946558619abc498998f6c559e8791687/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:29.231623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bcf85f-9f87-4233-a569-8263087df5df None admin] RBAC: Authorizing `identity:create_grant(project_id=7bfe28b4f0f648dd906926f955192736, group_id=946558619abc498998f6c559e8791687, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:29.233330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bcf85f-9f87-4233-a569-8263087df5df None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:29.246301 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:29.250218 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:29.254100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 208/403] 10.222.0.94 () {58 vars in 1442 bytes} [Mon Jul 18 13:55:29 2022] PUT /identity/v3/projects/7bfe28b4f0f648dd906926f955192736/groups/946558619abc498998f6c559e8791687/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:55:29.255718 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297d988-b2f0-4b8a-808d-10bbfc5c006e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:29.256184 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297d988-b2f0-4b8a-808d-10bbfc5c006e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:29.256630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297d988-b2f0-4b8a-808d-10bbfc5c006e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:29.258615 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 209/404] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:29.270280 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d272ef-138b-4396-a6ff-dc0921051a03 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:29.270836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d272ef-138b-4396-a6ff-dc0921051a03 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:29.271275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d272ef-138b-4396-a6ff-dc0921051a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:29.283578 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b43315-22a1-4395-87a8-2c6adcec7d49 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:29.307476 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d5d272ef-138b-4396-a6ff-dc0921051a03 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:29.308775 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d5d272ef-138b-4396-a6ff-dc0921051a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:29.331358 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b115b2c-b5aa-4e29-8f11-5c5abdf7d3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:29.332647 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b115b2c-b5aa-4e29-8f11-5c5abdf7d3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:29.333139 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b115b2c-b5aa-4e29-8f11-5c5abdf7d3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:29.333637 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b115b2c-b5aa-4e29-8f11-5c5abdf7d3d9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:29.346653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b115b2c-b5aa-4e29-8f11-5c5abdf7d3d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:29.355539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-3b115b2c-b5aa-4e29-8f11-5c5abdf7d3d9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:55:29.355539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:29.355539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:29.355539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:29.355539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:29.355539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:29.355539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:29.355539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:29.355539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:29.355539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:29.355539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:29.355539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:29.355539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:29.355539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:29.355539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:29.355539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:29.356540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:29.356540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:29.356540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:29.356540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:29.356540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:29.356540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:29.356540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:29.356540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:29.356540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:29.356540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:29.356540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:29.356540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:29.356540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:29.356540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:29.356540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:29.356540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:29.357316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:29.357316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:29.357316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:29.357316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:29.357316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:29.357316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:29.357316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:29.357316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:29.357316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:29.357316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:29.357316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:55:29.357316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:29.360158 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 196/405] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Jul 18 13:55:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:55:29.373065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b115b2c-b5aa-4e29-8f11-5c5abdf7d3d9 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:29.457326 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 210/406] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 18 13:55:29.483980 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d272ef-138b-4396-a6ff-dc0921051a03 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:29.524056 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b115b2c-b5aa-4e29-8f11-5c5abdf7d3d9 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:29.567762 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223b857-5d6a-48a4-9ed1-8ca771159634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:29.568491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223b857-5d6a-48a4-9ed1-8ca771159634 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:29.568779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223b857-5d6a-48a4-9ed1-8ca771159634 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:29.569110 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223b857-5d6a-48a4-9ed1-8ca771159634 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:29.570020 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6223b857-5d6a-48a4-9ed1-8ca771159634 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:29.570336 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6223b857-5d6a-48a4-9ed1-8ca771159634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:29.572321 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6223b857-5d6a-48a4-9ed1-8ca771159634 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:29.582078 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 197/407] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:29.587652 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223b857-5d6a-48a4-9ed1-8ca771159634 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:29.601142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:29.685946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d272ef-138b-4396-a6ff-dc0921051a03 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:29.701899 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223b857-5d6a-48a4-9ed1-8ca771159634 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:29.739184 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f28ae6-2133-4649-bae1-2e99eaa27055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:29.739996 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f28ae6-2133-4649-bae1-2e99eaa27055 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:29.740294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f28ae6-2133-4649-bae1-2e99eaa27055 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:29.740577 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f28ae6-2133-4649-bae1-2e99eaa27055 None admin] PATH_INFO: `/v3/groups/946558619abc498998f6c559e8791687` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:29.742182 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f28ae6-2133-4649-bae1-2e99eaa27055 None admin] RBAC: Authorizing `identity:get_group(group_id=946558619abc498998f6c559e8791687)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:29.743708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f28ae6-2133-4649-bae1-2e99eaa27055 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:29.745856 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 198/408] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Jul 18 13:55:29 2022] GET /identity/v3/groups/946558619abc498998f6c559e8791687 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:29.751208 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f28ae6-2133-4649-bae1-2e99eaa27055 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:29.816326 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:29.844326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:29.860076 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f28ae6-2133-4649-bae1-2e99eaa27055 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:29.872666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3b1ff-4002-4069-a022-2d6aba3f0f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:29.873492 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3b1ff-4002-4069-a022-2d6aba3f0f64 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:29.873787 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3b1ff-4002-4069-a022-2d6aba3f0f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:29.874073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3b1ff-4002-4069-a022-2d6aba3f0f64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:29.894729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3b1ff-4002-4069-a022-2d6aba3f0f64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:29.896933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6955f9-4615-437c-839f-6a0a4f1bb6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:29.897982 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6955f9-4615-437c-839f-6a0a4f1bb6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:29.898417 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6955f9-4615-437c-839f-6a0a4f1bb6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:29.898853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6955f9-4615-437c-839f-6a0a4f1bb6b4 None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:29.900312 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6955f9-4615-437c-839f-6a0a4f1bb6b4 None admin] RBAC: Authorizing `identity:get_project(project_id=7bfe28b4f0f648dd906926f955192736)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:29.901964 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6955f9-4615-437c-839f-6a0a4f1bb6b4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:29.904114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 199/409] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:29 2022] GET /identity/v3/projects/7bfe28b4f0f648dd906926f955192736 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:29.909667 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6955f9-4615-437c-839f-6a0a4f1bb6b4 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:29.910631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a2c3b1ff-4002-4069-a022-2d6aba3f0f64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:55:29.910631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:29.910631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:29.910631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:29.910631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:29.910631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:29.910631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:29.910631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:29.910631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:29.910631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:29.910631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:29.910631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:29.910631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:29.910631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:29.910631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:29.910631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:29.911719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:29.911719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:29.911719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:29.911719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:29.911719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:29.911719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:29.911719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:29.911719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:29.911719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:29.911719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:29.911719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:29.911719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:29.911719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:29.911719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:29.911719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:29.911719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:29.912761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:29.912761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:29.912761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:29.912761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:29.912761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:29.912761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:29.912761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:29.912761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:29.912761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:29.912761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:29.912761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:55:29.912761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:29.919686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 211/410] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:55:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 436 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:29.920586 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3b1ff-4002-4069-a022-2d6aba3f0f64 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:30.033382 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6955f9-4615-437c-839f-6a0a4f1bb6b4 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:30.067308 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3b1ff-4002-4069-a022-2d6aba3f0f64 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:30.077753 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb6633-9bea-4172-95e1-060293d95724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:30.079560 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb6633-9bea-4172-95e1-060293d95724 None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:30.080050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb6633-9bea-4172-95e1-060293d95724 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:30.080492 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb6633-9bea-4172-95e1-060293d95724 None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736/groups/946558619abc498998f6c559e8791687/roles/580aae575f414b3c9b4ccf1e27e0b640` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:30.083778 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb6633-9bea-4172-95e1-060293d95724 None admin] RBAC: Authorizing `identity:create_grant(project_id=7bfe28b4f0f648dd906926f955192736, group_id=946558619abc498998f6c559e8791687, role_id=580aae575f414b3c9b4ccf1e27e0b640)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:30.086487 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb6633-9bea-4172-95e1-060293d95724 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:30.104770 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 200/411] 10.222.0.94 () {58 vars in 1442 bytes} [Mon Jul 18 13:55:29 2022] PUT /identity/v3/projects/7bfe28b4f0f648dd906926f955192736/groups/946558619abc498998f6c559e8791687/roles/580aae575f414b3c9b4ccf1e27e0b640 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 13:55:30.109984 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536698b-d156-466e-9ceb-d4b01c79115c None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:30.109984 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536698b-d156-466e-9ceb-d4b01c79115c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:30.110229 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2536698b-d156-466e-9ceb-d4b01c79115c None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:30.113407 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 201/412] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:30.116990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3e58d-fe52-4d3b-9fb2-cc604f7b34a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:30.119291 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c668cb-9834-436c-a138-0100398f6df3 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:30.119591 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c668cb-9834-436c-a138-0100398f6df3 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:30.119873 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c668cb-9834-436c-a138-0100398f6df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:30.121556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3e58d-fe52-4d3b-9fb2-cc604f7b34a3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:30.121853 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3e58d-fe52-4d3b-9fb2-cc604f7b34a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:30.122134 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3e58d-fe52-4d3b-9fb2-cc604f7b34a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:30.122918 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3e58d-fe52-4d3b-9fb2-cc604f7b34a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:30.123232 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3e58d-fe52-4d3b-9fb2-cc604f7b34a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:30.125214 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3e58d-fe52-4d3b-9fb2-cc604f7b34a3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:30.134195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:30.145333 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 212/413] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:30.150094 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3e58d-fe52-4d3b-9fb2-cc604f7b34a3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:30.157930 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-47c668cb-9834-436c-a138-0100398f6df3 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:30.159297 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-47c668cb-9834-436c-a138-0100398f6df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:30.235366 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:30.295571 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3e58d-fe52-4d3b-9fb2-cc604f7b34a3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:30.330128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 202/414] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:30.335048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf9825-5958-434a-861f-5a51df2af87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:30.336297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf9825-5958-434a-861f-5a51df2af87e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:30.337363 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf9825-5958-434a-861f-5a51df2af87e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:30.337854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf9825-5958-434a-861f-5a51df2af87e None admin] PATH_INFO: `/v3/groups/e1306e19450043b4939c3d57f7452e3c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:30.339837 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf9825-5958-434a-861f-5a51df2af87e None admin] RBAC: Authorizing `identity:get_group(group_id=e1306e19450043b4939c3d57f7452e3c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:30.341459 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf9825-5958-434a-861f-5a51df2af87e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:30.343552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 213/415] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Jul 18 13:55:30 2022] GET /identity/v3/groups/e1306e19450043b4939c3d57f7452e3c => generated 229 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:30.348855 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c668cb-9834-436c-a138-0100398f6df3 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:30.357700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf9825-5958-434a-861f-5a51df2af87e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:30.463903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c668cb-9834-436c-a138-0100398f6df3 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:30.493872 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf9825-5958-434a-861f-5a51df2af87e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:30.499401 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d9bd7-ffef-4456-b3ff-a538d8d76762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:30.500393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d9bd7-ffef-4456-b3ff-a538d8d76762 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:30.500757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d9bd7-ffef-4456-b3ff-a538d8d76762 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:30.501154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d9bd7-ffef-4456-b3ff-a538d8d76762 None admin] PATH_INFO: `/v3/projects/defeadff0f9c417facade99f9905fca6` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:30.502467 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d9bd7-ffef-4456-b3ff-a538d8d76762 None admin] RBAC: Authorizing `identity:get_project(project_id=defeadff0f9c417facade99f9905fca6)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:30.504074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d9bd7-ffef-4456-b3ff-a538d8d76762 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:30.506175 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 203/416] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:30 2022] GET /identity/v3/projects/defeadff0f9c417facade99f9905fca6 => generated 344 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:30.512185 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d9bd7-ffef-4456-b3ff-a538d8d76762 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:30.515278 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:30.520193 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:30.544825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b277e3b-1e7c-4d12-b74d-73eda31eed81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:30.546710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b277e3b-1e7c-4d12-b74d-73eda31eed81 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:30.547222 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b277e3b-1e7c-4d12-b74d-73eda31eed81 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:30.548066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b277e3b-1e7c-4d12-b74d-73eda31eed81 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:30.563148 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b277e3b-1e7c-4d12-b74d-73eda31eed81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:30.573311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3b277e3b-1e7c-4d12-b74d-73eda31eed81 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:30.573311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:30.573311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:30.573311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:30.573311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:30.573311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:30.573311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:30.573311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:30.573311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:30.573311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:30.573311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:30.573311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:30.573311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:30.573311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:30.573311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:30.573311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:30.574272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:30.574272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:30.574272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:30.574272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:30.574272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:30.574272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:30.574272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:30.574272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:30.574272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:30.574272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:30.574272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:30.574272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:30.574272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:30.574272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:30.574272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:30.574272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:30.575069 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:30.575069 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:30.575069 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:30.575069 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:30.575069 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:30.575069 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:30.575069 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:30.575069 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:30.575069 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:30.575069 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:30.575069 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:30.575069 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:30.577929 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 214/417] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:55:30.584187 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b277e3b-1e7c-4d12-b74d-73eda31eed81 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:30.717181 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d9bd7-ffef-4456-b3ff-a538d8d76762 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:30.719456 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b277e3b-1e7c-4d12-b74d-73eda31eed81 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:30.764770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208edde9-01ab-42de-b762-256fe1b5080b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:30.766561 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208edde9-01ab-42de-b762-256fe1b5080b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:30.767454 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208edde9-01ab-42de-b762-256fe1b5080b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:30.768174 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208edde9-01ab-42de-b762-256fe1b5080b None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:30.769493 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208edde9-01ab-42de-b762-256fe1b5080b None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:30.770058 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208edde9-01ab-42de-b762-256fe1b5080b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:30.771640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208edde9-01ab-42de-b762-256fe1b5080b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:30.782679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 215/418] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:55:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:30.789202 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208edde9-01ab-42de-b762-256fe1b5080b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:30.799965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77922215-4827-48b7-ae0f-0872a0429278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:30.800806 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77922215-4827-48b7-ae0f-0872a0429278 None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:30.801102 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77922215-4827-48b7-ae0f-0872a0429278 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:30.801383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77922215-4827-48b7-ae0f-0872a0429278 None admin] PATH_INFO: `/v3/projects/defeadff0f9c417facade99f9905fca6/groups/e1306e19450043b4939c3d57f7452e3c/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:30.809452 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77922215-4827-48b7-ae0f-0872a0429278 None admin] RBAC: Authorizing `identity:create_grant(project_id=defeadff0f9c417facade99f9905fca6, group_id=e1306e19450043b4939c3d57f7452e3c, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:30.811312 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77922215-4827-48b7-ae0f-0872a0429278 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:30.838466 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 204/419] 10.222.0.94 () {58 vars in 1442 bytes} [Mon Jul 18 13:55:30 2022] PUT /identity/v3/projects/defeadff0f9c417facade99f9905fca6/groups/e1306e19450043b4939c3d57f7452e3c/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 13:55:30.845618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:30.904835 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208edde9-01ab-42de-b762-256fe1b5080b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:30.943522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b8fc8-8758-4dab-9a2b-6d1d1617ec6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:30.944605 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b8fc8-8758-4dab-9a2b-6d1d1617ec6d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:30.945126 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b8fc8-8758-4dab-9a2b-6d1d1617ec6d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:30.945611 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b8fc8-8758-4dab-9a2b-6d1d1617ec6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:31.023221 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b8fc8-8758-4dab-9a2b-6d1d1617ec6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:31.024812 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b8fc8-8758-4dab-9a2b-6d1d1617ec6d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:31.070259 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:31.132724 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 216/420] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:31.141054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77922215-4827-48b7-ae0f-0872a0429278 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:31.219349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:31.221739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1f92d-739b-45e3-8d17-d0cef43c8380 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:31.222116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1f92d-739b-45e3-8d17-d0cef43c8380 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:31.222403 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1f92d-739b-45e3-8d17-d0cef43c8380 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:31.223817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 217/421] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:31.230752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3600ac-e80b-40f7-a62f-d2533b394c7e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:31.231581 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3600ac-e80b-40f7-a62f-d2533b394c7e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:31.232095 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3600ac-e80b-40f7-a62f-d2533b394c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:31.259803 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0d3600ac-e80b-40f7-a62f-d2533b394c7e None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:31.261628 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0d3600ac-e80b-40f7-a62f-d2533b394c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:31.291767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77922215-4827-48b7-ae0f-0872a0429278 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:31.328068 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b77b9c-7ea9-4f44-8cae-6a37f8a69fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:31.329521 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b77b9c-7ea9-4f44-8cae-6a37f8a69fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:31.330036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b77b9c-7ea9-4f44-8cae-6a37f8a69fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:31.330513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b77b9c-7ea9-4f44-8cae-6a37f8a69fe2 None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:31.331896 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b77b9c-7ea9-4f44-8cae-6a37f8a69fe2 None admin] RBAC: Authorizing `identity:get_project(project_id=ec42cd390d9e4d07bb7573020b5b1003)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:31.333575 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b77b9c-7ea9-4f44-8cae-6a37f8a69fe2 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:31.335776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 205/422] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:31 2022] GET /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:31.342365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b77b9c-7ea9-4f44-8cae-6a37f8a69fe2 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:31.426631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 218/423] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 13:55:31.500648 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b77b9c-7ea9-4f44-8cae-6a37f8a69fe2 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:31.504981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3600ac-e80b-40f7-a62f-d2533b394c7e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:31.534160 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:31.541027 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d84129-4dd6-4ea5-a4c6-fc9abf1e0903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:31.542348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d84129-4dd6-4ea5-a4c6-fc9abf1e0903 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:31.542829 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d84129-4dd6-4ea5-a4c6-fc9abf1e0903 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:31.543294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d84129-4dd6-4ea5-a4c6-fc9abf1e0903 None admin] PATH_INFO: `/v3/groups/946558619abc498998f6c559e8791687` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:31.544688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d84129-4dd6-4ea5-a4c6-fc9abf1e0903 None admin] RBAC: Authorizing `identity:get_group(group_id=946558619abc498998f6c559e8791687)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:31.546451 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d84129-4dd6-4ea5-a4c6-fc9abf1e0903 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:31.548687 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 206/424] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Jul 18 13:55:31 2022] GET /identity/v3/groups/946558619abc498998f6c559e8791687 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:31.560978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d84129-4dd6-4ea5-a4c6-fc9abf1e0903 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:31.659801 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:31.716104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3600ac-e80b-40f7-a62f-d2533b394c7e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:31.749026 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d84129-4dd6-4ea5-a4c6-fc9abf1e0903 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:31.753136 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe8203-4ca2-4a88-8d92-e2deebdf6a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:31.753946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe8203-4ca2-4a88-8d92-e2deebdf6a39 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:31.754233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe8203-4ca2-4a88-8d92-e2deebdf6a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:31.754511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe8203-4ca2-4a88-8d92-e2deebdf6a39 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:31.765977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe8203-4ca2-4a88-8d92-e2deebdf6a39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:31.774064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-1cbe8203-4ca2-4a88-8d92-e2deebdf6a39 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:55:31.774064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:31.774064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:31.774064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:31.774064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:31.774064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:31.774064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:31.774064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:31.774064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:31.774064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:31.774064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:31.774064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:31.774064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:31.774064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:31.774064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:31.774064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:31.774928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:31.774928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:31.774928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:31.774928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:31.774928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:31.774928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:31.774928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:31.774928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:31.774928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:31.774928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:31.774928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:31.774928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:31.774928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:31.774928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:31.774928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:31.774928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:31.775723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:31.775723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:31.775723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:31.775723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:31.775723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:31.775723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:31.775723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:31.775723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:31.775723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:31.775723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:31.775723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:55:31.775723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:31.775723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 219/425] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Jul 18 13:55:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:55:31.776938 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba7d95-b955-4bf0-9794-50825ebcd40b None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:31.781739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba7d95-b955-4bf0-9794-50825ebcd40b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:31.781739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba7d95-b955-4bf0-9794-50825ebcd40b None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:31.781739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 220/426] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:31.784945 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba7d95-b955-4bf0-9794-50825ebcd40b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:31.792626 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78f99c-9642-49e1-b9be-ded21afcfc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:31.793375 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78f99c-9642-49e1-b9be-ded21afcfc54 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:31.793736 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78f99c-9642-49e1-b9be-ded21afcfc54 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:31.794084 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78f99c-9642-49e1-b9be-ded21afcfc54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:31.794987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78f99c-9642-49e1-b9be-ded21afcfc54 None admin] RBAC: Adding query filter params (group.id=946558619abc498998f6c559e8791687, role.id=1399a838106b4f9bb5fa1c942cc3ef22, scope.project.id=ec42cd390d9e4d07bb7573020b5b1003) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:31.795393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78f99c-9642-49e1-b9be-ded21afcfc54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:31.796943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78f99c-9642-49e1-b9be-ded21afcfc54 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:31.805939 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 207/427] 10.222.0.94 () {56 vars in 1376 bytes} [Mon Jul 18 13:55:31 2022] GET /identity/v3/role_assignments?group.id=946558619abc498998f6c559e8791687&scope.project.id=ec42cd390d9e4d07bb7573020b5b1003&role.id=1399a838106b4f9bb5fa1c942cc3ef22 => generated 611 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:31.807509 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacefd1-0851-494c-aee0-819790d16105 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:31.807866 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacefd1-0851-494c-aee0-819790d16105 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:31.808237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacefd1-0851-494c-aee0-819790d16105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:31.831157 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-2dacefd1-0851-494c-aee0-819790d16105 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:31.832529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2dacefd1-0851-494c-aee0-819790d16105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:31.968722 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:31.990629 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 208/428] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:31.993831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba7d95-b955-4bf0-9794-50825ebcd40b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:32.015654 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacefd1-0851-494c-aee0-819790d16105 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:32.031186 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:32.044715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1173526-b901-46b2-83a4-c5580f4af460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:32.045760 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1173526-b901-46b2-83a4-c5580f4af460 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:32.046238 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1173526-b901-46b2-83a4-c5580f4af460 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:32.046710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1173526-b901-46b2-83a4-c5580f4af460 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:32.047825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1173526-b901-46b2-83a4-c5580f4af460 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:32.049013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1173526-b901-46b2-83a4-c5580f4af460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:32.050775 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1173526-b901-46b2-83a4-c5580f4af460 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:32.068972 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 221/429] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:32.076439 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1173526-b901-46b2-83a4-c5580f4af460 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:32.142691 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacefd1-0851-494c-aee0-819790d16105 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:32.178643 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffa30e-e7ad-4c44-bfca-6ea2cf3b5a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:32.179590 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffa30e-e7ad-4c44-bfca-6ea2cf3b5a2c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:32.180012 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffa30e-e7ad-4c44-bfca-6ea2cf3b5a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:32.180928 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffa30e-e7ad-4c44-bfca-6ea2cf3b5a2c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:32.191886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffa30e-e7ad-4c44-bfca-6ea2cf3b5a2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:32.200506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-ceffa30e-e7ad-4c44-bfca-6ea2cf3b5a2c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:32.200506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:32.200506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:32.200506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:32.200506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:32.200506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:32.200506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:32.200506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:32.200506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:32.200506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:32.200506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:32.200506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:32.200506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:32.200506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:32.200506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:32.200506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:32.201381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:32.201381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:32.201381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:32.201381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:32.201381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:32.201381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:32.201381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:32.201381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:32.201381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:32.201381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:32.201381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:32.201381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:32.201381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:32.201381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:32.201381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:32.201381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:32.202191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:32.202191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:32.202191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:32.202191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:32.202191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:32.202191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:32.202191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:32.202191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:32.202191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:32.202191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:32.202191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 13:55:32.202191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:32.202191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 209/430] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:32.205573 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1173526-b901-46b2-83a4-c5580f4af460 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:32.207904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffa30e-e7ad-4c44-bfca-6ea2cf3b5a2c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:32.242577 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732bc5e-60dd-44e1-ab62-1a42147f1b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:32.244200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732bc5e-60dd-44e1-ab62-1a42147f1b6f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:32.244686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732bc5e-60dd-44e1-ab62-1a42147f1b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:32.245169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732bc5e-60dd-44e1-ab62-1a42147f1b6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:32.267063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:32.310591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:32.319166 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffa30e-e7ad-4c44-bfca-6ea2cf3b5a2c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:32.322685 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2732bc5e-60dd-44e1-ab62-1a42147f1b6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:32.324400 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2732bc5e-60dd-44e1-ab62-1a42147f1b6f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:32.355928 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d45169-f8b3-4c2c-8a67-326826af7d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:32.356779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d45169-f8b3-4c2c-8a67-326826af7d30 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:32.357133 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d45169-f8b3-4c2c-8a67-326826af7d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:32.357484 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d45169-f8b3-4c2c-8a67-326826af7d30 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:32.358315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d45169-f8b3-4c2c-8a67-326826af7d30 None admin] RBAC: Adding query filter params (name=member) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:32.358692 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d45169-f8b3-4c2c-8a67-326826af7d30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:32.360178 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d45169-f8b3-4c2c-8a67-326826af7d30 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:32.369582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 210/431] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:55:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:32.375343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d45169-f8b3-4c2c-8a67-326826af7d30 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:32.444287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 222/432] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:32.451542 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732bc5e-60dd-44e1-ab62-1a42147f1b6f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:32.505552 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d45169-f8b3-4c2c-8a67-326826af7d30 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:32.523679 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:32.549552 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126eed8-ed60-4bab-9547-a6723ba9a12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:32.550562 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126eed8-ed60-4bab-9547-a6723ba9a12d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:32.551112 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126eed8-ed60-4bab-9547-a6723ba9a12d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:32.551710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126eed8-ed60-4bab-9547-a6723ba9a12d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:32.562945 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:32.575576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732bc5e-60dd-44e1-ab62-1a42147f1b6f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:32.622387 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d56b3-e790-4efd-a62a-d6238fad85fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:32.624237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d56b3-e790-4efd-a62a-d6238fad85fa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:32.624708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d56b3-e790-4efd-a62a-d6238fad85fa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:32.625183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d56b3-e790-4efd-a62a-d6238fad85fa None admin] PATH_INFO: `/v3/projects/ec42cd390d9e4d07bb7573020b5b1003` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:32.627448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d56b3-e790-4efd-a62a-d6238fad85fa None admin] RBAC: Authorizing `identity:get_project(project_id=ec42cd390d9e4d07bb7573020b5b1003)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:32.629790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d56b3-e790-4efd-a62a-d6238fad85fa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:32.632336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 223/433] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:32 2022] GET /identity/v3/projects/ec42cd390d9e4d07bb7573020b5b1003 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:32.640949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d56b3-e790-4efd-a62a-d6238fad85fa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:32.642379 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b126eed8-ed60-4bab-9547-a6723ba9a12d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:32.643916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b126eed8-ed60-4bab-9547-a6723ba9a12d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:32.766441 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d56b3-e790-4efd-a62a-d6238fad85fa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:32.773214 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 211/434] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 13:55:32.775117 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f19ebe-22d3-4988-a202-4429f65367f3 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:32.775429 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f19ebe-22d3-4988-a202-4429f65367f3 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:32.775714 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f19ebe-22d3-4988-a202-4429f65367f3 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:32.777184 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 212/435] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:32.781891 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f19ebe-22d3-4988-a202-4429f65367f3 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:32.797181 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:32.809783 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6a61b-fac3-4f5c-bb43-11d2608afae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:32.810627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6a61b-fac3-4f5c-bb43-11d2608afae2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:32.810918 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6a61b-fac3-4f5c-bb43-11d2608afae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:32.811204 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6a61b-fac3-4f5c-bb43-11d2608afae2 None admin] PATH_INFO: `/v3/groups/946558619abc498998f6c559e8791687` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:32.812783 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6a61b-fac3-4f5c-bb43-11d2608afae2 None admin] RBAC: Authorizing `identity:get_group(group_id=946558619abc498998f6c559e8791687)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:32.814429 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6a61b-fac3-4f5c-bb43-11d2608afae2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:32.816271 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 224/436] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Jul 18 13:55:32 2022] GET /identity/v3/groups/946558619abc498998f6c559e8791687 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:32.822535 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b8caf-d9fd-432d-9d96-78360e02155d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:32.822824 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b8caf-d9fd-432d-9d96-78360e02155d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:32.823099 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b8caf-d9fd-432d-9d96-78360e02155d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:32.839518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-674b8caf-d9fd-432d-9d96-78360e02155d None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:32.840751 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-674b8caf-d9fd-432d-9d96-78360e02155d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:32.865111 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:32.903988 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f19ebe-22d3-4988-a202-4429f65367f3 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:32.923967 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 225/437] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:32.923967 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b8caf-d9fd-432d-9d96-78360e02155d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:32.939701 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7284d40-bf42-4cb8-8e00-081dc6b53354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:32.940642 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7284d40-bf42-4cb8-8e00-081dc6b53354 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:32.940928 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7284d40-bf42-4cb8-8e00-081dc6b53354 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:32.941266 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7284d40-bf42-4cb8-8e00-081dc6b53354 None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:32.942902 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7284d40-bf42-4cb8-8e00-081dc6b53354 None admin] RBAC: Authorizing `identity:get_project(project_id=7bfe28b4f0f648dd906926f955192736)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:32.946139 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7284d40-bf42-4cb8-8e00-081dc6b53354 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:32.952068 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 213/438] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:32 2022] GET /identity/v3/projects/7bfe28b4f0f648dd906926f955192736 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:32.952068 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7284d40-bf42-4cb8-8e00-081dc6b53354 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:33.037780 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b8caf-d9fd-432d-9d96-78360e02155d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:33.046514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:33.065891 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7284d40-bf42-4cb8-8e00-081dc6b53354 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:33.076048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b4d71-ef36-4f60-83ad-0c95b11bbeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:33.076726 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b4d71-ef36-4f60-83ad-0c95b11bbeec None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:33.077256 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b4d71-ef36-4f60-83ad-0c95b11bbeec None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:33.077604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b4d71-ef36-4f60-83ad-0c95b11bbeec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:33.078448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b4d71-ef36-4f60-83ad-0c95b11bbeec None admin] RBAC: Adding query filter params (group.id=946558619abc498998f6c559e8791687, role.id=580aae575f414b3c9b4ccf1e27e0b640, scope.project.id=ec42cd390d9e4d07bb7573020b5b1003) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:33.078768 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b4d71-ef36-4f60-83ad-0c95b11bbeec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:33.080595 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b4d71-ef36-4f60-83ad-0c95b11bbeec None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:33.089021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 226/439] 10.222.0.94 () {56 vars in 1376 bytes} [Mon Jul 18 13:55:32 2022] GET /identity/v3/role_assignments?group.id=946558619abc498998f6c559e8791687&scope.project.id=ec42cd390d9e4d07bb7573020b5b1003&role.id=580aae575f414b3c9b4ccf1e27e0b640 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:33.090116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b4d71-ef36-4f60-83ad-0c95b11bbeec None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:33.115981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5b02a-9ad0-40d8-af2b-a6ecf95ada5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:33.116988 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5b02a-9ad0-40d8-af2b-a6ecf95ada5d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:33.117394 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5b02a-9ad0-40d8-af2b-a6ecf95ada5d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:33.118314 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5b02a-9ad0-40d8-af2b-a6ecf95ada5d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:33.127439 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:33.133051 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a5b02a-9ad0-40d8-af2b-a6ecf95ada5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:33.142909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-55a5b02a-9ad0-40d8-af2b-a6ecf95ada5d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:55:33.142909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:33.142909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:33.142909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:33.142909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:33.142909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:33.142909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:33.142909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:33.142909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:33.142909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:33.142909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:33.142909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:33.142909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:33.142909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:33.142909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:33.142909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:33.144463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:33.144463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:33.144463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:33.144463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:33.144463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:33.144463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:33.144463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:33.144463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:33.144463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:33.144463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:33.144463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:33.144463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:33.144463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:33.144463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:33.144463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:33.144463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:33.145680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:33.145680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:33.145680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:33.145680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:33.145680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:33.145680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:33.145680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:33.145680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:33.145680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:33.145680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:33.145680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:55:33.145680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:33.145680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 214/440] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Jul 18 13:55:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:33.150507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5b02a-9ad0-40d8-af2b-a6ecf95ada5d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:33.226601 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b4d71-ef36-4f60-83ad-0c95b11bbeec None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:33.265176 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1073b4-3ddc-4d30-9ff5-87556a6cc18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:33.266045 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1073b4-3ddc-4d30-9ff5-87556a6cc18d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:33.266334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1073b4-3ddc-4d30-9ff5-87556a6cc18d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:33.266615 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1073b4-3ddc-4d30-9ff5-87556a6cc18d None admin] PATH_INFO: `/v3/groups/946558619abc498998f6c559e8791687` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:33.267811 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1073b4-3ddc-4d30-9ff5-87556a6cc18d None admin] RBAC: Authorizing `identity:get_group(group_id=946558619abc498998f6c559e8791687)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:33.269339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1073b4-3ddc-4d30-9ff5-87556a6cc18d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:33.271015 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 227/441] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Jul 18 13:55:33 2022] GET /identity/v3/groups/946558619abc498998f6c559e8791687 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:33.272174 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73294a43-bf56-45a5-aa2a-625cdcc64dd4 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:33.272533 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73294a43-bf56-45a5-aa2a-625cdcc64dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:33.272817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73294a43-bf56-45a5-aa2a-625cdcc64dd4 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:33.274233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 228/442] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:33.276995 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73294a43-bf56-45a5-aa2a-625cdcc64dd4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:33.279686 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5b02a-9ad0-40d8-af2b-a6ecf95ada5d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:33.323925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d587b6-f201-41ba-9492-1695b18628d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:33.324702 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d587b6-f201-41ba-9492-1695b18628d2 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:33.324835 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d587b6-f201-41ba-9492-1695b18628d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:33.325116 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d587b6-f201-41ba-9492-1695b18628d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:33.325868 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d587b6-f201-41ba-9492-1695b18628d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:33.326182 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d587b6-f201-41ba-9492-1695b18628d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:33.327573 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d587b6-f201-41ba-9492-1695b18628d2 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:33.336228 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 215/443] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:33.338199 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347731a3-e54f-4da7-8693-7d3d1e02153c None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:33.338493 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347731a3-e54f-4da7-8693-7d3d1e02153c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:33.338777 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347731a3-e54f-4da7-8693-7d3d1e02153c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:33.345616 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:33.357070 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-347731a3-e54f-4da7-8693-7d3d1e02153c None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:33.358281 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-347731a3-e54f-4da7-8693-7d3d1e02153c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:33.385252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:33.401352 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73294a43-bf56-45a5-aa2a-625cdcc64dd4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:33.439981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 216/444] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:33.440289 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347731a3-e54f-4da7-8693-7d3d1e02153c None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:33.449424 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e95ba-78ba-4a3d-bdeb-a031ca85d8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:33.450184 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e95ba-78ba-4a3d-bdeb-a031ca85d8ca None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:33.450542 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e95ba-78ba-4a3d-bdeb-a031ca85d8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:33.450901 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e95ba-78ba-4a3d-bdeb-a031ca85d8ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:33.451815 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e95ba-78ba-4a3d-bdeb-a031ca85d8ca None admin] RBAC: Adding query filter params (group.id=946558619abc498998f6c559e8791687, role.id=1399a838106b4f9bb5fa1c942cc3ef22, scope.project.id=7bfe28b4f0f648dd906926f955192736) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:33.452171 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e95ba-78ba-4a3d-bdeb-a031ca85d8ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:33.453639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e95ba-78ba-4a3d-bdeb-a031ca85d8ca None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:33.460554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 229/445] 10.222.0.94 () {56 vars in 1376 bytes} [Mon Jul 18 13:55:33 2022] GET /identity/v3/role_assignments?group.id=946558619abc498998f6c559e8791687&scope.project.id=7bfe28b4f0f648dd906926f955192736&role.id=1399a838106b4f9bb5fa1c942cc3ef22 => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:33.467408 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e95ba-78ba-4a3d-bdeb-a031ca85d8ca None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:33.572383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347731a3-e54f-4da7-8693-7d3d1e02153c None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:33.593482 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e95ba-78ba-4a3d-bdeb-a031ca85d8ca None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:33.612050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed01df-6ca8-4cd2-bb19-6e3d36e3513b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:33.612973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed01df-6ca8-4cd2-bb19-6e3d36e3513b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:33.613261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed01df-6ca8-4cd2-bb19-6e3d36e3513b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:33.613536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed01df-6ca8-4cd2-bb19-6e3d36e3513b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:33.617097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:33.632404 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f4710-1f47-4bab-90a0-7b9597fccd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:33.633206 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f4710-1f47-4bab-90a0-7b9597fccd39 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:33.634021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f4710-1f47-4bab-90a0-7b9597fccd39 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:33.634304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f4710-1f47-4bab-90a0-7b9597fccd39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:33.652344 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f4710-1f47-4bab-90a0-7b9597fccd39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:33.660093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f25f4710-1f47-4bab-90a0-7b9597fccd39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:55:33.660093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:33.660093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:33.660093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:33.660093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:33.660093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:33.660093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:33.660093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:33.660093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:33.660093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:33.660093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:33.660093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:33.660093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:33.660093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:33.660093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:33.660093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:33.660994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:33.660994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:33.660994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:33.660994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:33.660994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:33.660994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:33.660994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:33.660994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:33.660994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:33.660994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:33.660994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:33.660994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:33.660994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:33.660994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:33.660994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:33.660994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:33.661787 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:33.661787 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:33.661787 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:33.661787 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:33.661787 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:33.661787 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:33.661787 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:33.661787 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:33.661787 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:33.661787 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:33.661787 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:55:33.661787 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:33.661787 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 230/446] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:55:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:55:33.667787 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f4710-1f47-4bab-90a0-7b9597fccd39 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:33.684811 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:33.697129 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed01df-6ca8-4cd2-bb19-6e3d36e3513b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:33.698598 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed01df-6ca8-4cd2-bb19-6e3d36e3513b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:33.786748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f4710-1f47-4bab-90a0-7b9597fccd39 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:33.813501 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 217/447] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:33.819182 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed01df-6ca8-4cd2-bb19-6e3d36e3513b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:33.828889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec83ad-dc5f-4ae0-89d0-f5dba6e83002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:33.830082 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec83ad-dc5f-4ae0-89d0-f5dba6e83002 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:33.830199 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec83ad-dc5f-4ae0-89d0-f5dba6e83002 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:33.830389 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec83ad-dc5f-4ae0-89d0-f5dba6e83002 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:33.831174 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec83ad-dc5f-4ae0-89d0-f5dba6e83002 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:33.831910 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec83ad-dc5f-4ae0-89d0-f5dba6e83002 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:33.832954 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec83ad-dc5f-4ae0-89d0-f5dba6e83002 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:33.841573 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 231/448] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:33.847339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec83ad-dc5f-4ae0-89d0-f5dba6e83002 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:33.854538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:33.920484 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed01df-6ca8-4cd2-bb19-6e3d36e3513b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:33.923122 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:33.948820 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec83ad-dc5f-4ae0-89d0-f5dba6e83002 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:33.955281 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3038ba-02cb-4118-9ae1-41bb2a809433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:33.956171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3038ba-02cb-4118-9ae1-41bb2a809433 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:33.956455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3038ba-02cb-4118-9ae1-41bb2a809433 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:33.956732 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3038ba-02cb-4118-9ae1-41bb2a809433 None admin] PATH_INFO: `/v3/projects/7bfe28b4f0f648dd906926f955192736` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:33.957919 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3038ba-02cb-4118-9ae1-41bb2a809433 None admin] RBAC: Authorizing `identity:get_project(project_id=7bfe28b4f0f648dd906926f955192736)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:33.959361 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3038ba-02cb-4118-9ae1-41bb2a809433 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:33.961070 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 218/449] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:33 2022] GET /identity/v3/projects/7bfe28b4f0f648dd906926f955192736 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:33.967039 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3038ba-02cb-4118-9ae1-41bb2a809433 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:33.982704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36e6c5-3f9f-4a96-9e19-b51e688e77a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:33.983353 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36e6c5-3f9f-4a96-9e19-b51e688e77a7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:33.983645 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36e6c5-3f9f-4a96-9e19-b51e688e77a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:33.983928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36e6c5-3f9f-4a96-9e19-b51e688e77a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:34.049576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36e6c5-3f9f-4a96-9e19-b51e688e77a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:34.051028 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36e6c5-3f9f-4a96-9e19-b51e688e77a7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:34.063826 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3038ba-02cb-4118-9ae1-41bb2a809433 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:34.065379 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:34.101673 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d861f2-d90e-4a13-8994-952eaf82d432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:34.102589 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d861f2-d90e-4a13-8994-952eaf82d432 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:34.102874 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d861f2-d90e-4a13-8994-952eaf82d432 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:34.103151 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d861f2-d90e-4a13-8994-952eaf82d432 None admin] PATH_INFO: `/v3/groups/946558619abc498998f6c559e8791687` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:34.104349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d861f2-d90e-4a13-8994-952eaf82d432 None admin] RBAC: Authorizing `identity:get_group(group_id=946558619abc498998f6c559e8791687)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:34.105787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d861f2-d90e-4a13-8994-952eaf82d432 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:34.107391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 219/450] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Jul 18 13:55:33 2022] GET /identity/v3/groups/946558619abc498998f6c559e8791687 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:34.113173 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d861f2-d90e-4a13-8994-952eaf82d432 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:34.125918 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:34.150272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 232/451] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 13:55:34.155728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36e6c5-3f9f-4a96-9e19-b51e688e77a7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:34.213187 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d861f2-d90e-4a13-8994-952eaf82d432 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:34.247292 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f8c3e-1b58-4ff9-a2fe-7478dfa04be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:34.247971 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f8c3e-1b58-4ff9-a2fe-7478dfa04be4 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:34.248264 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f8c3e-1b58-4ff9-a2fe-7478dfa04be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:34.248547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f8c3e-1b58-4ff9-a2fe-7478dfa04be4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:34.249371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f8c3e-1b58-4ff9-a2fe-7478dfa04be4 None admin] RBAC: Adding query filter params (group.id=946558619abc498998f6c559e8791687, role.id=580aae575f414b3c9b4ccf1e27e0b640, scope.project.id=7bfe28b4f0f648dd906926f955192736) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:34.249688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f8c3e-1b58-4ff9-a2fe-7478dfa04be4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:34.251127 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f8c3e-1b58-4ff9-a2fe-7478dfa04be4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:34.252417 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36e6c5-3f9f-4a96-9e19-b51e688e77a7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:34.256782 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 220/452] 10.222.0.94 () {56 vars in 1376 bytes} [Mon Jul 18 13:55:34 2022] GET /identity/v3/role_assignments?group.id=946558619abc498998f6c559e8791687&scope.project.id=7bfe28b4f0f648dd906926f955192736&role.id=580aae575f414b3c9b4ccf1e27e0b640 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:34.287092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee8a04f-12df-4919-b557-73e5d0c5b6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:34.287994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8a04f-12df-4919-b557-73e5d0c5b6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:34.288284 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8a04f-12df-4919-b557-73e5d0c5b6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:34.288559 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8a04f-12df-4919-b557-73e5d0c5b6d7 None admin] PATH_INFO: `/v3/projects/defeadff0f9c417facade99f9905fca6` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:34.290386 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee8a04f-12df-4919-b557-73e5d0c5b6d7 None admin] RBAC: Authorizing `identity:get_project(project_id=defeadff0f9c417facade99f9905fca6)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:34.291886 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee8a04f-12df-4919-b557-73e5d0c5b6d7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:34.293690 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 233/453] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:55:34 2022] GET /identity/v3/projects/defeadff0f9c417facade99f9905fca6 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:34.299339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f8c3e-1b58-4ff9-a2fe-7478dfa04be4 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:34.335839 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:34.404983 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f8c3e-1b58-4ff9-a2fe-7478dfa04be4 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:34.438601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b426f-0e61-40b4-867e-ed8cbfa1f985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:34.439458 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b426f-0e61-40b4-867e-ed8cbfa1f985 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:34.439740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b426f-0e61-40b4-867e-ed8cbfa1f985 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:34.440042 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b426f-0e61-40b4-867e-ed8cbfa1f985 None admin] PATH_INFO: `/v3/groups/e1306e19450043b4939c3d57f7452e3c` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:34.441206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26b426f-0e61-40b4-867e-ed8cbfa1f985 None admin] RBAC: Authorizing `identity:get_group(group_id=e1306e19450043b4939c3d57f7452e3c)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:34.442634 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26b426f-0e61-40b4-867e-ed8cbfa1f985 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:34.444269 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 221/454] 10.222.0.94 () {56 vars in 1180 bytes} [Mon Jul 18 13:55:34 2022] GET /identity/v3/groups/e1306e19450043b4939c3d57f7452e3c => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:34.450162 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee8a04f-12df-4919-b557-73e5d0c5b6d7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:34.503901 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:34.558707 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee8a04f-12df-4919-b557-73e5d0c5b6d7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:34.594376 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8a86c-9dbd-4182-a886-e6e39b6550e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:34.595039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8a86c-9dbd-4182-a886-e6e39b6550e0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:34.595326 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8a86c-9dbd-4182-a886-e6e39b6550e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:34.595608 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8a86c-9dbd-4182-a886-e6e39b6550e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:34.596525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8a86c-9dbd-4182-a886-e6e39b6550e0 None admin] RBAC: Adding query filter params (group.id=e1306e19450043b4939c3d57f7452e3c, role.id=c60a99de086f45c6820f7872a6b43feb, scope.project.id=defeadff0f9c417facade99f9905fca6) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:34.596840 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8a86c-9dbd-4182-a886-e6e39b6550e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:34.598292 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8a86c-9dbd-4182-a886-e6e39b6550e0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:34.604290 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 234/455] 10.222.0.94 () {56 vars in 1376 bytes} [Mon Jul 18 13:55:34 2022] GET /identity/v3/role_assignments?group.id=e1306e19450043b4939c3d57f7452e3c&scope.project.id=defeadff0f9c417facade99f9905fca6&role.id=c60a99de086f45c6820f7872a6b43feb => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:37.041650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2836d-4648-4d22-a710-0a494c4905a6 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:37.042181 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2836d-4648-4d22-a710-0a494c4905a6 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:37.042181 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2836d-4648-4d22-a710-0a494c4905a6 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:37.043807 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 222/456] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:37.049150 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05e062-ff0c-455a-b117-a69b4bcacc1c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:37.049595 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05e062-ff0c-455a-b117-a69b4bcacc1c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:37.049800 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05e062-ff0c-455a-b117-a69b4bcacc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:37.076977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-da05e062-ff0c-455a-b117-a69b4bcacc1c None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:37.078186 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-da05e062-ff0c-455a-b117-a69b4bcacc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:37.174556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 235/457] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:37.206551 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2836d-4648-4d22-a710-0a494c4905a6 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:37.333602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2836d-4648-4d22-a710-0a494c4905a6 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:37.335153 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:37.383908 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9a13a-3352-444f-83f0-28b317b5c7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:37.384753 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9a13a-3352-444f-83f0-28b317b5c7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:37.385041 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9a13a-3352-444f-83f0-28b317b5c7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:37.385308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9a13a-3352-444f-83f0-28b317b5c7e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:37.396186 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e9a13a-3352-444f-83f0-28b317b5c7e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:37.397725 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e9a13a-3352-444f-83f0-28b317b5c7e8 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:37.416167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-a1e9a13a-3352-444f-83f0-28b317b5c7e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:37.416167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:37.416167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:37.416167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:37.416167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:37.416167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:37.416167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:37.416167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:37.416167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:37.416167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:37.416167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:37.416167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:37.416167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:37.416167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:37.416167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:37.417097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:37.417097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:37.417097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:37.417097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:37.417097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:37.417097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:37.417097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:37.417097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:37.417097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:37.417097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:37.417097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:37.417097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:37.417097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:37.417097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:37.417097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:37.417097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:37.417097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:37.417945 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:37.417945 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:37.417945 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:37.417945 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:37.417945 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 223/458] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:37.423759 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05e062-ff0c-455a-b117-a69b4bcacc1c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:37.473051 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519effec-a28d-4c72-a52b-f177d42eba74 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:37.473345 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519effec-a28d-4c72-a52b-f177d42eba74 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:37.473539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519effec-a28d-4c72-a52b-f177d42eba74 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:37.475161 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 224/459] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:55:37.481264 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdda27-a6b2-46e5-a493-ad988f688a4e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:37.481555 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdda27-a6b2-46e5-a493-ad988f688a4e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:37.481836 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdda27-a6b2-46e5-a493-ad988f688a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:37.490788 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:37.503692 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-6fcdda27-a6b2-46e5-a493-ad988f688a4e None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:37.505226 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6fcdda27-a6b2-46e5-a493-ad988f688a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:37.557699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05e062-ff0c-455a-b117-a69b4bcacc1c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:37.577489 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 225/460] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:37.579419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadcd0f-5437-4b5d-8fc7-70f1538cbcf8 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:37.580056 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadcd0f-5437-4b5d-8fc7-70f1538cbcf8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:37.580530 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadcd0f-5437-4b5d-8fc7-70f1538cbcf8 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:37.582234 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 226/461] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:37.591996 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e811ed-be81-4f50-b4e7-72659f822a39 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:37.592499 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e811ed-be81-4f50-b4e7-72659f822a39 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:37.593135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e811ed-be81-4f50-b4e7-72659f822a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:37.594270 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42e2c3-a7c0-467c-b05b-836594f31020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:37.595154 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42e2c3-a7c0-467c-b05b-836594f31020 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:37.595582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42e2c3-a7c0-467c-b05b-836594f31020 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:37.596033 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42e2c3-a7c0-467c-b05b-836594f31020 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:37.596905 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42e2c3-a7c0-467c-b05b-836594f31020 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:37.597520 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42e2c3-a7c0-467c-b05b-836594f31020 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:37.599210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42e2c3-a7c0-467c-b05b-836594f31020 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:37.613203 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 236/462] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:37.614319 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42e2c3-a7c0-467c-b05b-836594f31020 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:37.622030 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-03e811ed-be81-4f50-b4e7-72659f822a39 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:37.623217 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-03e811ed-be81-4f50-b4e7-72659f822a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:37.700537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 227/463] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:37.705063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e811ed-be81-4f50-b4e7-72659f822a39 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:37.720902 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:37.733697 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:37.738112 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42e2c3-a7c0-467c-b05b-836594f31020 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:37.774914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3ecbe-e753-47a9-8518-65f9e91880dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:37.775975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3ecbe-e753-47a9-8518-65f9e91880dc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:37.776274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3ecbe-e753-47a9-8518-65f9e91880dc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:37.776552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3ecbe-e753-47a9-8518-65f9e91880dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:37.787159 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3ecbe-e753-47a9-8518-65f9e91880dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:37.788722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3ecbe-e753-47a9-8518-65f9e91880dc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:37.806781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c8e3ecbe-e753-47a9-8518-65f9e91880dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:37.806781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:37.806781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:37.806781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:37.806781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:37.806781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:37.806781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:37.806781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:37.806781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:37.806781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:37.806781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:37.806781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:37.806781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:37.806781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:37.806781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:37.807644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:37.807644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:37.807644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:37.807644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:37.807644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:37.807644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:37.807644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:37.807644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:37.807644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:37.807644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:37.807644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:37.807644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:37.807644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:37.807644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:37.807644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:37.807644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:37.807644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:37.808527 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:37.808527 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:37.808527 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:37.808527 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:37.808527 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 237/464] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:37.809305 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65dbc4-54bf-45bf-ac28-2c72d35e4c7f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:37.809466 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65dbc4-54bf-45bf-ac28-2c72d35e4c7f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:37.809886 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65dbc4-54bf-45bf-ac28-2c72d35e4c7f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:37.815357 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 238/465] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:37.815357 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65dbc4-54bf-45bf-ac28-2c72d35e4c7f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:37.815572 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e811ed-be81-4f50-b4e7-72659f822a39 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:37.851410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3e5af-1419-4892-8387-dc3762c057c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:37.852588 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3e5af-1419-4892-8387-dc3762c057c1 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:37.852876 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3e5af-1419-4892-8387-dc3762c057c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:37.853232 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3e5af-1419-4892-8387-dc3762c057c1 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:37.853897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3e5af-1419-4892-8387-dc3762c057c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:37.855339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3e5af-1419-4892-8387-dc3762c057c1 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:37.876198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 228/466] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Jul 18 13:55:37 2022] POST /identity/v3/users => generated 246 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:55:37.877310 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3e5af-1419-4892-8387-dc3762c057c1 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:37.956293 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:37.970823 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65dbc4-54bf-45bf-ac28-2c72d35e4c7f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:38.000401 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3e5af-1419-4892-8387-dc3762c057c1 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:38.015899 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce9e1a-8784-497d-bc6c-fca62b6d454a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:38.016850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce9e1a-8784-497d-bc6c-fca62b6d454a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:38.017149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce9e1a-8784-497d-bc6c-fca62b6d454a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:38.017436 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce9e1a-8784-497d-bc6c-fca62b6d454a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:38.029938 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce9e1a-8784-497d-bc6c-fca62b6d454a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:38.031565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce9e1a-8784-497d-bc6c-fca62b6d454a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:38.042223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67af165-82cc-42d3-a4dd-5d582408f318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:38.043648 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67af165-82cc-42d3-a4dd-5d582408f318 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:38.044019 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67af165-82cc-42d3-a4dd-5d582408f318 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:38.044287 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67af165-82cc-42d3-a4dd-5d582408f318 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:38.044969 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67af165-82cc-42d3-a4dd-5d582408f318 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:38.045275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67af165-82cc-42d3-a4dd-5d582408f318 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:38.046641 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67af165-82cc-42d3-a4dd-5d582408f318 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:38.053334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-cdce9e1a-8784-497d-bc6c-fca62b6d454a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:38.053334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:38.053334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:38.053334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:38.053334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:38.053334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:38.053334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:38.053334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:38.053334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:38.053334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:38.053334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:38.053334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:38.053334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:38.053334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:38.053334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:38.054464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:38.054464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:38.054464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:38.054464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:38.054464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:38.054464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:38.054464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:38.054464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:38.054464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:38.054464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:38.054464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:38.054464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:38.054464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:38.054464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:38.054464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:38.054464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:38.054464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:38.056462 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:38.056462 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:38.056462 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:38.056462 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:38.056462 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 239/467] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:38.056742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7900d78-bba8-4914-9ac3-97163f81229a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:38.057066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7900d78-bba8-4914-9ac3-97163f81229a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:38.058350 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7900d78-bba8-4914-9ac3-97163f81229a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:38.062594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:38.073706 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 229/468] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:38.073706 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67af165-82cc-42d3-a4dd-5d582408f318 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:38.081691 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a7900d78-bba8-4914-9ac3-97163f81229a None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:38.083233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a7900d78-bba8-4914-9ac3-97163f81229a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:38.178277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 240/469] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:38.179164 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7900d78-bba8-4914-9ac3-97163f81229a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:38.216309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67af165-82cc-42d3-a4dd-5d582408f318 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:38.259620 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:38.269153 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5fdc1e-f9de-45cc-ad21-25ed360f79a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:38.270817 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fdc1e-f9de-45cc-ad21-25ed360f79a9 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:38.271344 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fdc1e-f9de-45cc-ad21-25ed360f79a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:38.271829 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fdc1e-f9de-45cc-ad21-25ed360f79a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:38.272791 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5fdc1e-f9de-45cc-ad21-25ed360f79a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:38.273871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5fdc1e-f9de-45cc-ad21-25ed360f79a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:38.275780 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5fdc1e-f9de-45cc-ad21-25ed360f79a9 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:38.285963 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7900d78-bba8-4914-9ac3-97163f81229a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:38.292436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 230/470] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:38.293486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5fdc1e-f9de-45cc-ad21-25ed360f79a9 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:38.324772 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:38.331408 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7ea8d-d419-444d-beaa-ca5f798531e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:38.332114 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7ea8d-d419-444d-beaa-ca5f798531e0 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:38.332400 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7ea8d-d419-444d-beaa-ca5f798531e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:38.332679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7ea8d-d419-444d-beaa-ca5f798531e0 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:38.333576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7ea8d-d419-444d-beaa-ca5f798531e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:38.335159 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7ea8d-d419-444d-beaa-ca5f798531e0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:38.360700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 241/471] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Jul 18 13:55:38 2022] POST /identity/v3/users => generated 248 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:55:38.361559 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7ea8d-d419-444d-beaa-ca5f798531e0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:38.425500 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5fdc1e-f9de-45cc-ad21-25ed360f79a9 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:38.466004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8724e9d-d914-4f6d-911d-a300f92415ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:38.467367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8724e9d-d914-4f6d-911d-a300f92415ce None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:38.467863 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8724e9d-d914-4f6d-911d-a300f92415ce None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:38.468393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8724e9d-d914-4f6d-911d-a300f92415ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:38.476096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7ea8d-d419-444d-beaa-ca5f798531e0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:38.484355 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8724e9d-d914-4f6d-911d-a300f92415ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:38.485867 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8724e9d-d914-4f6d-911d-a300f92415ce None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:38.505696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-e8724e9d-d914-4f6d-911d-a300f92415ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:38.505696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:38.505696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:38.505696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:38.505696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:38.505696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:38.505696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:38.505696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:38.505696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:38.505696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:38.505696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:38.505696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:38.505696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:38.505696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:38.505696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:38.506608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:38.506608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:38.506608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:38.506608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:38.506608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:38.506608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:38.506608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:38.506608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:38.506608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:38.506608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:38.506608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:38.506608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:38.506608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:38.506608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:38.506608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:38.506608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:38.506608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:38.507991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:38.507991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:38.507991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:38.507991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:38.507991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 231/472] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:38.509036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edbd47-27ce-44b7-8014-a72f91e51cd5 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:38.509209 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edbd47-27ce-44b7-8014-a72f91e51cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:38.509800 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edbd47-27ce-44b7-8014-a72f91e51cd5 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:38.511287 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 232/473] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:38.515290 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edbd47-27ce-44b7-8014-a72f91e51cd5 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:38.516302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61195174-8805-4a32-8882-c2ada1d28ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:38.516981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61195174-8805-4a32-8882-c2ada1d28ad1 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:38.517262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61195174-8805-4a32-8882-c2ada1d28ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:38.517539 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61195174-8805-4a32-8882-c2ada1d28ad1 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:38.518244 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61195174-8805-4a32-8882-c2ada1d28ad1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:38.519980 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61195174-8805-4a32-8882-c2ada1d28ad1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:38.549138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 242/474] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Jul 18 13:55:38 2022] POST /identity/v3/users => generated 248 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:55:38.550914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba829c-0bea-4b93-b0ed-2c5c9ca2d6e4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:38.551488 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba829c-0bea-4b93-b0ed-2c5c9ca2d6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:38.551927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba829c-0bea-4b93-b0ed-2c5c9ca2d6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:38.575809 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4eba829c-0bea-4b93-b0ed-2c5c9ca2d6e4 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:38.577972 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4eba829c-0bea-4b93-b0ed-2c5c9ca2d6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:38.582715 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:38.645091 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:38.661853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edbd47-27ce-44b7-8014-a72f91e51cd5 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:38.682809 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 243/475] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:38.714729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba829c-0bea-4b93-b0ed-2c5c9ca2d6e4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:38.723402 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87bf055-f02a-4040-a4f6-46aedf781e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:38.724446 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bf055-f02a-4040-a4f6-46aedf781e43 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:38.724996 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bf055-f02a-4040-a4f6-46aedf781e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:38.725497 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bf055-f02a-4040-a4f6-46aedf781e43 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:38.726483 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87bf055-f02a-4040-a4f6-46aedf781e43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:38.727526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87bf055-f02a-4040-a4f6-46aedf781e43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:38.729267 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87bf055-f02a-4040-a4f6-46aedf781e43 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:38.743673 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 233/476] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:38.750351 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87bf055-f02a-4040-a4f6-46aedf781e43 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:38.853198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba829c-0bea-4b93-b0ed-2c5c9ca2d6e4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:38.873118 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87bf055-f02a-4040-a4f6-46aedf781e43 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:38.875135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:38.902315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e3255-5dd8-489c-b522-c546a425263b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:38.903005 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e3255-5dd8-489c-b522-c546a425263b None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:38.903308 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e3255-5dd8-489c-b522-c546a425263b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:38.903989 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e3255-5dd8-489c-b522-c546a425263b None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:38.904645 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e3255-5dd8-489c-b522-c546a425263b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:38.906762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e3255-5dd8-489c-b522-c546a425263b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:38.916464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.sql.core [None req-e04e3255-5dd8-489c-b522-c546a425263b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 13:55:38.916464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 13:55:38.916464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [parameters: {'id': 'eac921e4827c465f9ada0a91017afa32', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 13:55:38.916464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80160) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 13:55:38.918921 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbfe9a-0fa9-47b6-94cd-735fd8f6e346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:38.919645 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbfe9a-0fa9-47b6-94cd-735fd8f6e346 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:38.920002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbfe9a-0fa9-47b6-94cd-735fd8f6e346 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:38.920290 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbfe9a-0fa9-47b6-94cd-735fd8f6e346 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:38.920997 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dbfe9a-0fa9-47b6-94cd-735fd8f6e346 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:38.922578 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dbfe9a-0fa9-47b6-94cd-735fd8f6e346 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:38.938210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e04e3255-5dd8-489c-b522-c546a425263b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 13:55:38.938210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:38.938210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 13:55:38.938210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 13:55:38.938210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 13:55:38.938210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 13:55:38.938210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 13:55:38.938210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 13:55:38.938210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 13:55:38.938210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application conn.query(q) Jul 18 13:55:38.938210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 13:55:38.938210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 13:55:38.938210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 13:55:38.938210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result.read() Jul 18 13:55:38.938210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 13:55:38.939125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 13:55:38.939125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 13:55:38.939125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 13:55:38.939125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 13:55:38.939125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 13:55:38.939125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 13:55:38.939125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 13:55:38.939125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 13:55:38.939125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:38.939125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 13:55:38.939125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:38.939125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:38.939125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 13:55:38.939125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 13:55:38.939125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 18 13:55:38.939125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return ref.to_dict() Jul 18 13:55:38.939125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 13:55:38.939125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application next(self.gen) Jul 18 13:55:38.940547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 13:55:38.940547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application yield resource Jul 18 13:55:38.940547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 13:55:38.940547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application next(self.gen) Jul 18 13:55:38.940547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 13:55:38.940547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 13:55:38.940547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 13:55:38.940547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 13:55:38.940547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 13:55:38.940547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise self.value Jul 18 13:55:38.940547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 13:55:38.940547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 13:55:38.940547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 13:55:38.940547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application session.commit() Jul 18 13:55:38.940547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 13:55:38.940547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 13:55:38.940547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 13:55:38.942679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 13:55:38.942679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 13:55:38.942679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.session.flush() Jul 18 13:55:38.942679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 13:55:38.942679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 13:55:38.942679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 13:55:38.942679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 13:55:38.942679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 13:55:38.942679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application compat.raise_( Jul 18 13:55:38.942679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 13:55:38.942679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception Jul 18 13:55:38.942679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 13:55:38.942679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 13:55:38.942679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 13:55:38.942679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 13:55:38.942679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 13:55:38.942679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 13:55:38.952051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 13:55:38.952051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 13:55:38.952051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 13:55:38.952051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 13:55:38.952051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 13:55:38.952051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 13:55:38.952051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 13:55:38.952051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 13:55:38.952051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 13:55:38.952051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 13:55:38.952051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 13:55:38.952051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 13:55:38.952051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 13:55:38.952051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 13:55:38.952051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 13:55:38.952051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception Jul 18 13:55:38.954891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 13:55:38.954891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 13:55:38.954891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 13:55:38.954891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 13:55:38.954891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 13:55:38.954891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 13:55:38.954891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 13:55:38.954891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application conn.query(q) Jul 18 13:55:38.954891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 13:55:38.954891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 13:55:38.954891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 13:55:38.954891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result.read() Jul 18 13:55:38.954891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 13:55:38.954891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 13:55:38.954891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 13:55:38.954891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 13:55:38.956508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 13:55:38.956508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 13:55:38.956508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 13:55:38.956508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 13:55:38.956508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 13:55:38.956508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 13:55:38.956508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [parameters: {'id': 'eac921e4827c465f9ada0a91017afa32', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 13:55:38.956508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 13:55:38.956508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:38.956508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:55:38.956508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:38.956508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:38.956508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:38.956508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:38.956508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:38.956508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:38.958013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:38.958013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:38.958013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:38.958013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:38.958013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:38.958013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:38.958013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 18 13:55:38.958013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 18 13:55:38.958013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:38.958013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:38.958013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 18 13:55:38.958013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 18 13:55:38.958013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 13:55:38.958013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 13:55:38.958013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 13:55:38.958013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:38.960240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 244/477] 10.222.0.94 () {60 vars in 1131 bytes} [Mon Jul 18 13:55:38 2022] POST /identity/v3/roles => generated 144 bytes in 226 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:38.960240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e3255-5dd8-489c-b522-c546a425263b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:38.960369 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 234/478] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Jul 18 13:55:38 2022] POST /identity/v3/users => generated 249 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:55:39.003458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:39.097001 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e3255-5dd8-489c-b522-c546a425263b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:39.154314 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e5570-0418-4834-b1dd-9ef17e806926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:39.155154 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e5570-0418-4834-b1dd-9ef17e806926 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:39.155442 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e5570-0418-4834-b1dd-9ef17e806926 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:39.155722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e5570-0418-4834-b1dd-9ef17e806926 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:39.175423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e5570-0418-4834-b1dd-9ef17e806926 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:39.184151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-0d2e5570-0418-4834-b1dd-9ef17e806926 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:55:39.184151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:39.184151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:39.184151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:39.184151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:39.184151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:39.184151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:39.184151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:39.184151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:39.184151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:39.184151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:39.184151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:39.184151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:39.184151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:39.184151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:39.184151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:39.186233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:39.186233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:39.186233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:39.186233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:39.186233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:39.186233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:39.186233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:39.186233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:39.186233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:39.186233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:39.186233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:39.186233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:39.186233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:39.186233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:39.186233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:39.186233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:39.188262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:39.188262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:39.188262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:39.188262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:39.188262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:39.188262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:39.188262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:39.188262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:39.188262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:39.188262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:39.188262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 13:55:39.188262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:39.188262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 245/479] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Jul 18 13:55:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:39.193066 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbfe9a-0fa9-47b6-94cd-735fd8f6e346 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:39.317824 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbfe9a-0fa9-47b6-94cd-735fd8f6e346 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:39.355495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42217983-064a-419e-ad66-8b74da7943ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:39.356204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42217983-064a-419e-ad66-8b74da7943ce None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:39.356497 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42217983-064a-419e-ad66-8b74da7943ce None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:39.356779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42217983-064a-419e-ad66-8b74da7943ce None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:39.357601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42217983-064a-419e-ad66-8b74da7943ce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:39.357851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42217983-064a-419e-ad66-8b74da7943ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:39.359228 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42217983-064a-419e-ad66-8b74da7943ce None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:39.368173 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 235/480] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:40.310749 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cecbe-2fb0-4070-96ec-a3b442b22eba None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:40.311425 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cecbe-2fb0-4070-96ec-a3b442b22eba None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:40.311425 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15cecbe-2fb0-4070-96ec-a3b442b22eba None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:40.312864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 246/481] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:40.320126 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fa9d9-3cb2-438a-af87-06364c2104ca None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:40.320126 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fa9d9-3cb2-438a-af87-06364c2104ca None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:40.320126 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fa9d9-3cb2-438a-af87-06364c2104ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:40.342042 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-fb4fa9d9-3cb2-438a-af87-06364c2104ca None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:40.343511 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fb4fa9d9-3cb2-438a-af87-06364c2104ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:40.360822 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:40.430729 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 236/482] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:40.466124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15cecbe-2fb0-4070-96ec-a3b442b22eba None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:40.608520 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:40.618088 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15cecbe-2fb0-4070-96ec-a3b442b22eba None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:40.657939 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee2093-7398-489e-b7f9-acda6823e17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:40.658733 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee2093-7398-489e-b7f9-acda6823e17c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:40.659022 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee2093-7398-489e-b7f9-acda6823e17c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:40.659299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee2093-7398-489e-b7f9-acda6823e17c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:40.676755 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee2093-7398-489e-b7f9-acda6823e17c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:40.684881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-d4ee2093-7398-489e-b7f9-acda6823e17c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:40.684881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:40.684881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:40.684881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:40.684881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:40.684881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:40.684881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:40.684881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:40.684881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:40.684881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:40.684881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:40.684881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:40.684881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:40.684881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:40.684881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:40.684881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:40.685782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:40.685782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:40.685782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:40.685782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:40.685782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:40.685782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:40.685782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:40.685782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:40.685782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:40.685782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:40.685782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:40.685782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:40.685782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:40.685782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:40.685782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:40.685782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:40.687124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:40.687124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:40.687124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:40.687124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:40.687124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:40.687124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:40.687124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:40.687124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:40.687124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:40.687124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:40.687124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:40.687124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:40.687124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 247/483] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:55:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:40.693911 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fa9d9-3cb2-438a-af87-06364c2104ca None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:40.832327 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fa9d9-3cb2-438a-af87-06364c2104ca None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:40.869337 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06d2d6-10f3-46e4-a1ae-8450e233faf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:40.870351 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06d2d6-10f3-46e4-a1ae-8450e233faf2 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:40.870854 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06d2d6-10f3-46e4-a1ae-8450e233faf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:40.871328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06d2d6-10f3-46e4-a1ae-8450e233faf2 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:40.872318 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06d2d6-10f3-46e4-a1ae-8450e233faf2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:40.872823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06d2d6-10f3-46e4-a1ae-8450e233faf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:40.874396 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06d2d6-10f3-46e4-a1ae-8450e233faf2 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:40.885660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e35fa2-00f2-443d-b3da-d7ee1786ce14 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:40.885965 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e35fa2-00f2-443d-b3da-d7ee1786ce14 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:40.886240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e35fa2-00f2-443d-b3da-d7ee1786ce14 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:40.887854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 248/485] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:55:40.887989 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 237/485] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Jul 18 13:55:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:40.893541 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e35fa2-00f2-443d-b3da-d7ee1786ce14 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:40.904576 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bb461-c32f-4ef6-a15a-dc8e513bf074 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:40.905202 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bb461-c32f-4ef6-a15a-dc8e513bf074 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:40.905704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bb461-c32f-4ef6-a15a-dc8e513bf074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:40.925518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-112bb461-c32f-4ef6-a15a-dc8e513bf074 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:40.927728 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-112bb461-c32f-4ef6-a15a-dc8e513bf074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:40.948318 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:41.077687 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e35fa2-00f2-443d-b3da-d7ee1786ce14 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:41.077969 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 238/486] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 13:55:41.087054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc8537-b850-49aa-8cc0-02dbdf5428ca None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:41.087351 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc8537-b850-49aa-8cc0-02dbdf5428ca None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:41.087637 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc8537-b850-49aa-8cc0-02dbdf5428ca None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:41.089348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 239/487] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:55:41.094378 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59210874-21d2-4c95-87be-fe43340872c4 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:41.094676 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59210874-21d2-4c95-87be-fe43340872c4 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:41.094960 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59210874-21d2-4c95-87be-fe43340872c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:41.120412 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94d38f-4c80-427a-a592-73c142ca5e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:41.121128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94d38f-4c80-427a-a592-73c142ca5e12 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:41.121418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94d38f-4c80-427a-a592-73c142ca5e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:41.121704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94d38f-4c80-427a-a592-73c142ca5e12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:41.125371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-59210874-21d2-4c95-87be-fe43340872c4 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:41.126783 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-59210874-21d2-4c95-87be-fe43340872c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:41.132981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94d38f-4c80-427a-a592-73c142ca5e12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:41.135065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94d38f-4c80-427a-a592-73c142ca5e12 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:41.153380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-0b94d38f-4c80-427a-a592-73c142ca5e12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:41.153380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:41.153380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:41.153380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:41.153380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:41.153380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:41.153380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:41.153380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:41.153380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:41.153380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:41.153380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:41.153380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:41.153380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:41.153380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:41.153380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:41.154224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:41.154224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:41.154224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:41.154224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:41.154224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:41.154224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:41.154224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:41.154224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:41.154224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:41.154224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:41.154224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:41.154224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:41.154224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:41.154224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:41.154224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:41.154224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:41.154224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:41.155446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:41.155446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:41.155446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:41.155446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:41.155446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 249/488] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:41.155446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94d38f-4c80-427a-a592-73c142ca5e12 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:41.172888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:41.212807 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 240/489] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:41.214460 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59210874-21d2-4c95-87be-fe43340872c4 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:41.283508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94d38f-4c80-427a-a592-73c142ca5e12 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:41.295439 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:41.328491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59210874-21d2-4c95-87be-fe43340872c4 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:41.332931 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453ee52-c850-4526-9f17-b72f1f29e951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:41.334332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453ee52-c850-4526-9f17-b72f1f29e951 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:41.334845 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453ee52-c850-4526-9f17-b72f1f29e951 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:41.335322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453ee52-c850-4526-9f17-b72f1f29e951 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:41.350093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1453ee52-c850-4526-9f17-b72f1f29e951 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:41.361999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-1453ee52-c850-4526-9f17-b72f1f29e951 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:41.361999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:41.361999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:41.361999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:41.361999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:41.361999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:41.361999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:41.361999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:41.361999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:41.361999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:41.361999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:41.361999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:41.361999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:41.361999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:41.361999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:41.361999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:41.363059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:41.363059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:41.363059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:41.363059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:41.363059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:41.363059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:41.363059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:41.363059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:41.363059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:41.363059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:41.363059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:41.363059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:41.363059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:41.363059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:41.363059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:41.363059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:41.364077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:41.364077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:41.364077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:41.364077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:41.364077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:41.364077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:41.364077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:41.364077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:41.364077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:41.364077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:41.364077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:41.364077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:41.370519 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 250/490] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:55:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:41.370519 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453ee52-c850-4526-9f17-b72f1f29e951 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:41.375009 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cbf0d-ab90-4e6d-bb84-19c7571047de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:41.375851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cbf0d-ab90-4e6d-bb84-19c7571047de None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:41.376167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cbf0d-ab90-4e6d-bb84-19c7571047de None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:41.376613 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cbf0d-ab90-4e6d-bb84-19c7571047de None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:41.377359 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3cbf0d-ab90-4e6d-bb84-19c7571047de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:41.378429 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3cbf0d-ab90-4e6d-bb84-19c7571047de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:41.380803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3cbf0d-ab90-4e6d-bb84-19c7571047de None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:41.398373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 241/491] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:41.400087 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cbf0d-ab90-4e6d-bb84-19c7571047de None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:41.463385 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:41.505480 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453ee52-c850-4526-9f17-b72f1f29e951 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:41.515828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cbf0d-ab90-4e6d-bb84-19c7571047de None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:41.553491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:41.557049 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde22ba-34db-4565-b689-dc82403574a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:41.558304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde22ba-34db-4565-b689-dc82403574a8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:41.559096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde22ba-34db-4565-b689-dc82403574a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:41.559566 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde22ba-34db-4565-b689-dc82403574a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:41.559675 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3dc507-1332-4c40-8d6a-9a1ec9178f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:41.560422 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3dc507-1332-4c40-8d6a-9a1ec9178f87 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:41.561140 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3dc507-1332-4c40-8d6a-9a1ec9178f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:41.561746 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3dc507-1332-4c40-8d6a-9a1ec9178f87 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:41.562727 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3dc507-1332-4c40-8d6a-9a1ec9178f87 None admin] RBAC: Adding query filter params (name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:41.563140 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3dc507-1332-4c40-8d6a-9a1ec9178f87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:41.564811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3dc507-1332-4c40-8d6a-9a1ec9178f87 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:41.574076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde22ba-34db-4565-b689-dc82403574a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:41.578951 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 242/492] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Jul 18 13:55:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:41.578951 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3dc507-1332-4c40-8d6a-9a1ec9178f87 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:41.589663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-afde22ba-34db-4565-b689-dc82403574a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:41.589663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:41.589663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:41.589663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:41.589663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:41.589663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:41.589663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:41.589663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:41.589663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:41.589663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:41.589663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:41.589663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:41.589663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:41.589663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:41.589663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:41.589663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:41.590750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:41.590750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:41.590750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:41.590750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:41.590750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:41.590750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:41.590750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:41.590750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:41.590750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:41.590750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:41.590750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:41.590750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:41.590750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:41.590750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:41.590750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:41.590750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:41.591737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:41.591737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:41.591737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:41.591737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:41.591737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:41.591737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:41.591737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:41.591737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:41.591737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:41.591737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:41.591737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:41.591737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:41.594312 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 251/493] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:55:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:41.595764 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde22ba-34db-4565-b689-dc82403574a8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:41.700396 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3dc507-1332-4c40-8d6a-9a1ec9178f87 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:41.724534 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde22ba-34db-4565-b689-dc82403574a8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:41.744123 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d4be4-dab1-4608-8cee-087cda7613b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:41.745074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d4be4-dab1-4608-8cee-087cda7613b4 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:41.745463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d4be4-dab1-4608-8cee-087cda7613b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:41.745844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d4be4-dab1-4608-8cee-087cda7613b4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:41.756067 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d4be4-dab1-4608-8cee-087cda7613b4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:41.758368 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d4be4-dab1-4608-8cee-087cda7613b4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:41.759677 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:41.767885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d80b74-12a8-49ea-8e7d-f313125b7efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:41.768972 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d80b74-12a8-49ea-8e7d-f313125b7efe None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:41.769427 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d80b74-12a8-49ea-8e7d-f313125b7efe None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:41.769872 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d80b74-12a8-49ea-8e7d-f313125b7efe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:41.778529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-5f5d4be4-dab1-4608-8cee-087cda7613b4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 13:55:41.778529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:41.778529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:41.778529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:41.778529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:41.778529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:41.778529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:41.778529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:41.778529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:41.778529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:41.778529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:41.778529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:41.778529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:55:41.778529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:55:41.778529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:55:41.778529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:55:41.779471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:41.779471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:41.779471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:55:41.779471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:41.779471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:55:41.779471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:41.779471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:41.779471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:41.779471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:41.779471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:41.779471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:41.779471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:41.779471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:41.779471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:41.779471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:41.779471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:41.779471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:41.779471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:41.780716 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:41.780716 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:41.780716 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:55:41.780716 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:55:41.780716 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:55:41.780716 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:55:41.780716 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:55:41.780716 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:55:41.780716 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 13:55:41.780716 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:41.784019 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 243/494] 10.222.0.94 () {56 vars in 1093 bytes} [Mon Jul 18 13:55:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:41.785012 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d4be4-dab1-4608-8cee-087cda7613b4 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:41.787709 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d80b74-12a8-49ea-8e7d-f313125b7efe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:41.790274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d80b74-12a8-49ea-8e7d-f313125b7efe None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:41.812576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-40d80b74-12a8-49ea-8e7d-f313125b7efe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:41.812576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:41.812576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:41.812576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:41.812576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:41.812576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:41.812576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:41.812576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:41.812576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:41.812576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:41.812576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:41.812576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:41.812576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:41.812576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:41.812576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:41.813458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:41.813458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:41.813458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:41.813458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:41.813458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:41.813458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:41.813458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:41.813458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:41.813458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:41.813458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:41.813458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:41.813458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:41.813458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:41.813458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:41.813458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:41.813458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:41.813458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:41.814307 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:41.814307 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:41.814307 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:41.814307 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:41.816355 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 252/495] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:41.817797 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d80b74-12a8-49ea-8e7d-f313125b7efe None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:41.850191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:41.900225 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d4be4-dab1-4608-8cee-087cda7613b4 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:41.926242 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d80b74-12a8-49ea-8e7d-f313125b7efe None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:41.940377 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c55191-b414-4317-8c37-ea9c473ec88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:41.941051 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c55191-b414-4317-8c37-ea9c473ec88c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:41.941334 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c55191-b414-4317-8c37-ea9c473ec88c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:41.941616 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c55191-b414-4317-8c37-ea9c473ec88c None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:41.942362 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c55191-b414-4317-8c37-ea9c473ec88c None admin] RBAC: Adding query filter params (name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:41.943056 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c55191-b414-4317-8c37-ea9c473ec88c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:41.946339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c55191-b414-4317-8c37-ea9c473ec88c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:41.960368 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 244/496] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Jul 18 13:55:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:41.960368 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ae917-05af-4342-89f6-a2f38515bd59 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:41.960368 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ae917-05af-4342-89f6-a2f38515bd59 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:41.960368 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ae917-05af-4342-89f6-a2f38515bd59 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:41.964276 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8e699-28ac-4079-9174-ff7e9c42e412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:41.964822 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 245/497] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:41.965021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8e699-28ac-4079-9174-ff7e9c42e412 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:41.965365 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8e699-28ac-4079-9174-ff7e9c42e412 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:41.965590 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ae917-05af-4342-89f6-a2f38515bd59 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:41.965683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8e699-28ac-4079-9174-ff7e9c42e412 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:41.966523 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8e699-28ac-4079-9174-ff7e9c42e412 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:41.966843 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8e699-28ac-4079-9174-ff7e9c42e412 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:41.968310 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8e699-28ac-4079-9174-ff7e9c42e412 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:42.005346 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 253/498] 10.222.0.94 () {56 vars in 1133 bytes} [Mon Jul 18 13:55:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:42.006172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8e699-28ac-4079-9174-ff7e9c42e412 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:42.046833 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:42.074780 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ae917-05af-4342-89f6-a2f38515bd59 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:42.091220 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:42.103919 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8e699-28ac-4079-9174-ff7e9c42e412 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:42.109681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed4975-e365-4eba-919f-8038353efb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:42.110341 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed4975-e365-4eba-919f-8038353efb58 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:42.110625 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed4975-e365-4eba-919f-8038353efb58 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:42.110904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed4975-e365-4eba-919f-8038353efb58 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:42.111617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed4975-e365-4eba-919f-8038353efb58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:42.111923 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed4975-e365-4eba-919f-8038353efb58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:42.113319 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed4975-e365-4eba-919f-8038353efb58 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:42.124858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 246/499] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:42.126026 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd3a54-83e4-4854-af7e-7e3791b41af4 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:42.126313 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd3a54-83e4-4854-af7e-7e3791b41af4 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:42.126591 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd3a54-83e4-4854-af7e-7e3791b41af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:42.136643 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c32bf2-9d54-4ac6-839d-fb72ad5d66ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:42.137425 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c32bf2-9d54-4ac6-839d-fb72ad5d66ee None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:42.137709 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c32bf2-9d54-4ac6-839d-fb72ad5d66ee None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:42.137986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c32bf2-9d54-4ac6-839d-fb72ad5d66ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:42.142079 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-9cbd3a54-83e4-4854-af7e-7e3791b41af4 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:42.143261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9cbd3a54-83e4-4854-af7e-7e3791b41af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:42.148245 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c32bf2-9d54-4ac6-839d-fb72ad5d66ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:42.150270 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c32bf2-9d54-4ac6-839d-fb72ad5d66ee None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:42.168171 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-39c32bf2-9d54-4ac6-839d-fb72ad5d66ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:42.168171 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:42.168171 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:42.168171 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:42.168171 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:42.168171 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:42.168171 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:42.168171 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:42.168171 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:42.168171 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:42.168171 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:42.168171 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:42.168171 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:42.168171 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:42.168171 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:42.169100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:42.169100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:42.169100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:42.169100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:42.169100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:42.169100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:42.169100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:42.169100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:42.169100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:42.169100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:42.169100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:42.169100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:42.169100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:42.169100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:42.169100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:42.169100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:42.169100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:42.169942 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:42.169942 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:42.169942 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:42.169942 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:42.169942 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 254/500] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:42.170237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c32bf2-9d54-4ac6-839d-fb72ad5d66ee None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:42.224913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 247/501] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:42.226160 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc27a8-076b-43de-802e-d2cd12d2bded None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:42.226452 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc27a8-076b-43de-802e-d2cd12d2bded None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:42.226731 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc27a8-076b-43de-802e-d2cd12d2bded None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:42.228217 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 248/502] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:42.228757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc27a8-076b-43de-802e-d2cd12d2bded None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:42.267855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:42.277836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c32bf2-9d54-4ac6-839d-fb72ad5d66ee None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:42.312278 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c7b19-913a-4088-b1d2-0cceaab1bff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:42.312925 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c7b19-913a-4088-b1d2-0cceaab1bff1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:42.313216 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c7b19-913a-4088-b1d2-0cceaab1bff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:42.313502 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c7b19-913a-4088-b1d2-0cceaab1bff1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:42.338173 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc27a8-076b-43de-802e-d2cd12d2bded None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:42.345408 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:42.372249 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3def6-2976-4f26-90ed-e3c0f64d0395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:42.373043 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3def6-2976-4f26-90ed-e3c0f64d0395 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:42.373331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3def6-2976-4f26-90ed-e3c0f64d0395 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:42.373607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3def6-2976-4f26-90ed-e3c0f64d0395 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:42.380169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c7b19-913a-4088-b1d2-0cceaab1bff1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:42.380599 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3def6-2976-4f26-90ed-e3c0f64d0395 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:42.381656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c7b19-913a-4088-b1d2-0cceaab1bff1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:42.381840 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3def6-2976-4f26-90ed-e3c0f64d0395 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:42.396535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-f4f3def6-2976-4f26-90ed-e3c0f64d0395 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 13:55:42.396535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:42.396535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:42.396535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:42.396535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:42.396535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:42.396535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:42.396535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:42.396535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:42.396535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:42.396535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:42.396535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:42.396535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:55:42.396535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:55:42.396535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:55:42.396535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:55:42.397568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:42.397568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:42.397568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:55:42.397568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:42.397568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:55:42.397568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:42.397568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:42.397568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:42.397568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:42.397568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:42.397568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:42.397568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:42.397568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:42.397568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:42.397568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:42.397568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:42.397568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:42.397568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:42.398740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:42.398740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:42.398740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:55:42.398740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:55:42.398740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:55:42.398740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:55:42.398740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:55:42.398740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:55:42.398740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 13:55:42.398740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:42.398740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 249/503] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:42.398740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3def6-2976-4f26-90ed-e3c0f64d0395 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:42.473213 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:42.483097 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 255/504] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:42.484325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712b54c-3613-496d-88b5-2806e03ee92a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:42.484620 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712b54c-3613-496d-88b5-2806e03ee92a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:42.484904 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712b54c-3613-496d-88b5-2806e03ee92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:42.496001 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3def6-2976-4f26-90ed-e3c0f64d0395 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:42.500515 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b712b54c-3613-496d-88b5-2806e03ee92a None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:42.501713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b712b54c-3613-496d-88b5-2806e03ee92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:42.529492 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791f6fa-2532-4813-b812-411025272865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:42.530149 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791f6fa-2532-4813-b812-411025272865 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:42.530431 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791f6fa-2532-4813-b812-411025272865 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:42.530703 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791f6fa-2532-4813-b812-411025272865 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:42.531413 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791f6fa-2532-4813-b812-411025272865 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:42.531715 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791f6fa-2532-4813-b812-411025272865 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:42.533102 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791f6fa-2532-4813-b812-411025272865 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:42.544026 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 250/505] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:42.544813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791f6fa-2532-4813-b812-411025272865 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:42.568715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 256/506] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:42.569547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712b54c-3613-496d-88b5-2806e03ee92a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:42.572721 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:42.648338 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791f6fa-2532-4813-b812-411025272865 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:42.675049 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712b54c-3613-496d-88b5-2806e03ee92a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:42.682022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792db28b-642b-40a3-955a-83ade0dee6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:42.683203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792db28b-642b-40a3-955a-83ade0dee6cf None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:42.683498 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792db28b-642b-40a3-955a-83ade0dee6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:42.683776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792db28b-642b-40a3-955a-83ade0dee6cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:42.694588 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792db28b-642b-40a3-955a-83ade0dee6cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:42.697618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:42.702255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-792db28b-642b-40a3-955a-83ade0dee6cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:42.702255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:42.702255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:42.702255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:42.702255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:42.702255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:42.702255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:42.702255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:42.702255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:42.702255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:42.702255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:42.702255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:42.702255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:42.702255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:42.702255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:42.702255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:42.703136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:42.703136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:42.703136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:42.703136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:42.703136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:42.703136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:42.703136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:42.703136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:42.703136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:42.703136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:42.703136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:42.703136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:42.703136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:42.703136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:42.703136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:42.703136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:42.703985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:42.703985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:42.703985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:42.703985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:42.703985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:42.703985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:42.703985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:42.703985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:42.703985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:42.703985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:42.703985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:42.703985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:42.703985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 251/507] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:55:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:42.704911 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792db28b-642b-40a3-955a-83ade0dee6cf None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:42.714534 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edf021-4d99-43af-9ef9-2d598b56417f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:42.715203 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edf021-4d99-43af-9ef9-2d598b56417f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:42.715489 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edf021-4d99-43af-9ef9-2d598b56417f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:42.715767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edf021-4d99-43af-9ef9-2d598b56417f None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:42.716646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66edf021-4d99-43af-9ef9-2d598b56417f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:42.716958 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66edf021-4d99-43af-9ef9-2d598b56417f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:42.718394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66edf021-4d99-43af-9ef9-2d598b56417f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:42.735247 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 257/508] 10.222.0.94 () {56 vars in 1137 bytes} [Mon Jul 18 13:55:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:42.735985 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edf021-4d99-43af-9ef9-2d598b56417f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:42.788092 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:42.803833 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792db28b-642b-40a3-955a-83ade0dee6cf None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:42.833735 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edf021-4d99-43af-9ef9-2d598b56417f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:42.836776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b412d1-cf03-4751-9a31-cb280559a7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:42.837584 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b412d1-cf03-4751-9a31-cb280559a7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:42.837874 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b412d1-cf03-4751-9a31-cb280559a7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:42.838153 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b412d1-cf03-4751-9a31-cb280559a7b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:42.848492 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b412d1-cf03-4751-9a31-cb280559a7b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:42.849954 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b412d1-cf03-4751-9a31-cb280559a7b2 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:42.867204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-40b412d1-cf03-4751-9a31-cb280559a7b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:42.867204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:42.867204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:42.867204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:42.867204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:42.867204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:42.867204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:42.867204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:42.867204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:42.867204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:42.867204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:42.867204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:42.867204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:42.867204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:42.867204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:42.868260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:42.868260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:42.868260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:42.868260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:42.868260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:42.868260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:42.868260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:42.868260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:42.868260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:42.868260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:42.868260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:42.868260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:42.868260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:42.868260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:42.868260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:42.868260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:42.868260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:42.869094 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:42.869094 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:42.869094 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:42.869094 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:42.869314 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09d4ec-3e51-4dab-a6a9-6415d74cfbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:42.869314 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09d4ec-3e51-4dab-a6a9-6415d74cfbfb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:42.869314 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09d4ec-3e51-4dab-a6a9-6415d74cfbfb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:42.869504 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 252/509] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:42.869578 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09d4ec-3e51-4dab-a6a9-6415d74cfbfb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:42.870209 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b412d1-cf03-4751-9a31-cb280559a7b2 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:42.876558 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09d4ec-3e51-4dab-a6a9-6415d74cfbfb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:42.878014 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09d4ec-3e51-4dab-a6a9-6415d74cfbfb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:42.893692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6a09d4ec-3e51-4dab-a6a9-6415d74cfbfb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 13:55:42.893692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:42.893692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:42.893692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:42.893692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:42.893692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:42.893692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:42.893692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:42.893692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:42.893692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:42.893692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:42.893692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:42.893692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:55:42.893692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:55:42.893692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:55:42.893692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:55:42.894791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:42.894791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:42.894791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:55:42.894791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:42.894791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:55:42.894791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:42.894791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:42.894791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:42.894791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:42.894791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:42.894791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:42.894791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:42.894791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:42.894791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:42.894791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:42.894791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:42.894791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:42.894791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:42.895713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:42.895713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:42.895713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:55:42.895713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:55:42.895713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:55:42.895713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:55:42.895713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:55:42.895713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:55:42.895713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 13:55:42.895713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:42.895713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 258/510] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:42.895713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09d4ec-3e51-4dab-a6a9-6415d74cfbfb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:42.941060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:42.985725 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b412d1-cf03-4751-9a31-cb280559a7b2 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:43.002456 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09d4ec-3e51-4dab-a6a9-6415d74cfbfb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:43.019471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a7d6a-99de-4cc8-8187-f2ef74f85656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:43.020300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a7d6a-99de-4cc8-8187-f2ef74f85656 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:43.020589 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a7d6a-99de-4cc8-8187-f2ef74f85656 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:43.020868 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a7d6a-99de-4cc8-8187-f2ef74f85656 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:43.027280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:43.032416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a7d6a-99de-4cc8-8187-f2ef74f85656 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:43.033867 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a7d6a-99de-4cc8-8187-f2ef74f85656 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:43.035388 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffa8f1-36ac-4fe1-9bc1-188f5f3e22f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:43.036089 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffa8f1-36ac-4fe1-9bc1-188f5f3e22f7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:43.036439 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffa8f1-36ac-4fe1-9bc1-188f5f3e22f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:43.036730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffa8f1-36ac-4fe1-9bc1-188f5f3e22f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:43.037512 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbffa8f1-36ac-4fe1-9bc1-188f5f3e22f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:43.037831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbffa8f1-36ac-4fe1-9bc1-188f5f3e22f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:43.039208 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbffa8f1-36ac-4fe1-9bc1-188f5f3e22f7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:43.047586 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 259/511] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Jul 18 13:55:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:43.048405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffa8f1-36ac-4fe1-9bc1-188f5f3e22f7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:43.051865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-2d9a7d6a-99de-4cc8-8187-f2ef74f85656 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:43.051865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:43.051865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:43.051865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:43.051865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:43.051865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:43.051865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:43.051865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:43.051865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:43.051865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:43.051865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:43.051865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:43.051865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:43.051865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:43.051865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:43.052978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:43.052978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:43.052978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:43.052978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:43.052978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:43.052978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:43.052978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:43.052978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:43.052978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:43.052978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:43.052978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:43.052978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:43.052978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:43.052978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:43.052978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:43.052978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:43.052978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:43.054032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:43.054032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:43.054032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:43.054032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:43.054032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 253/512] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:43.054422 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a7d6a-99de-4cc8-8187-f2ef74f85656 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:43.149446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffa8f1-36ac-4fe1-9bc1-188f5f3e22f7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:43.150931 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:43.151768 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a7d6a-99de-4cc8-8187-f2ef74f85656 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:43.182666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17523130-5c82-4d8f-a6e3-bf9d5f4cac24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:43.183306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17523130-5c82-4d8f-a6e3-bf9d5f4cac24 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:43.183596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17523130-5c82-4d8f-a6e3-bf9d5f4cac24 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:43.183780 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86baca1b-b49e-4a5d-8713-b47cb820b946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:43.183886 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17523130-5c82-4d8f-a6e3-bf9d5f4cac24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:43.184485 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86baca1b-b49e-4a5d-8713-b47cb820b946 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:43.184768 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86baca1b-b49e-4a5d-8713-b47cb820b946 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:43.185045 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86baca1b-b49e-4a5d-8713-b47cb820b946 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:43.185758 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86baca1b-b49e-4a5d-8713-b47cb820b946 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:43.186064 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86baca1b-b49e-4a5d-8713-b47cb820b946 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:43.187422 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86baca1b-b49e-4a5d-8713-b47cb820b946 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:43.199259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 254/513] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:43.200061 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86baca1b-b49e-4a5d-8713-b47cb820b946 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:43.253438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:43.256986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17523130-5c82-4d8f-a6e3-bf9d5f4cac24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:43.259010 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17523130-5c82-4d8f-a6e3-bf9d5f4cac24 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:43.313355 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86baca1b-b49e-4a5d-8713-b47cb820b946 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:43.348411 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988e5ba-9296-4968-8c5c-5596cbf0eda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:43.349723 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988e5ba-9296-4968-8c5c-5596cbf0eda0 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:43.350008 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988e5ba-9296-4968-8c5c-5596cbf0eda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:43.350289 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988e5ba-9296-4968-8c5c-5596cbf0eda0 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:43.351128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1988e5ba-9296-4968-8c5c-5596cbf0eda0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:43.351446 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1988e5ba-9296-4968-8c5c-5596cbf0eda0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:43.353492 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1988e5ba-9296-4968-8c5c-5596cbf0eda0 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:43.358091 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 260/514] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:43.358929 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17523130-5c82-4d8f-a6e3-bf9d5f4cac24 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:43.361333 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:43.390581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 255/515] 10.222.0.94 () {56 vars in 1137 bytes} [Mon Jul 18 13:55:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:43.391247 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988e5ba-9296-4968-8c5c-5596cbf0eda0 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:43.458284 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17523130-5c82-4d8f-a6e3-bf9d5f4cac24 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:43.489479 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988e5ba-9296-4968-8c5c-5596cbf0eda0 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:43.491365 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3c1dc-7430-45e2-b8d5-1b78cb47a0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:43.492184 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3c1dc-7430-45e2-b8d5-1b78cb47a0dd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:43.492490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3c1dc-7430-45e2-b8d5-1b78cb47a0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:43.492767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3c1dc-7430-45e2-b8d5-1b78cb47a0dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:43.498544 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:43.502454 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c3c1dc-7430-45e2-b8d5-1b78cb47a0dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:43.503965 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c3c1dc-7430-45e2-b8d5-1b78cb47a0dd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:43.520808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-78c3c1dc-7430-45e2-b8d5-1b78cb47a0dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:43.520808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:43.520808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:43.520808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:43.520808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:43.520808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:43.520808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:43.520808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:43.520808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:43.520808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:43.520808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:43.520808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:43.520808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:43.520808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:43.520808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:43.521650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:43.521650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:43.521650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:43.521650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:43.521650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:43.521650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:43.521650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:43.521650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:43.521650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:43.521650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:43.521650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:43.521650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:43.521650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:43.521650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:43.521650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:43.521650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:43.521650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:43.522506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:43.522506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:43.522506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:43.522506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:43.522506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 261/516] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:43.522783 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3c1dc-7430-45e2-b8d5-1b78cb47a0dd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:43.523609 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fb980-f36c-441d-a43e-f7acddbccc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:43.525037 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fb980-f36c-441d-a43e-f7acddbccc16 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:43.525322 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fb980-f36c-441d-a43e-f7acddbccc16 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:43.525599 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fb980-f36c-441d-a43e-f7acddbccc16 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:43.526320 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132fb980-f36c-441d-a43e-f7acddbccc16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:43.526628 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132fb980-f36c-441d-a43e-f7acddbccc16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:43.528042 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132fb980-f36c-441d-a43e-f7acddbccc16 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:43.539425 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 256/517] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:43.540656 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fb980-f36c-441d-a43e-f7acddbccc16 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:43.580669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:43.625186 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3c1dc-7430-45e2-b8d5-1b78cb47a0dd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:43.644104 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fb980-f36c-441d-a43e-f7acddbccc16 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:43.659349 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217eb9de-c90a-45b2-a3fd-3407fec157d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:43.660167 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217eb9de-c90a-45b2-a3fd-3407fec157d1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:43.660450 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217eb9de-c90a-45b2-a3fd-3407fec157d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:43.660723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217eb9de-c90a-45b2-a3fd-3407fec157d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:43.667742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217eb9de-c90a-45b2-a3fd-3407fec157d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:43.669228 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217eb9de-c90a-45b2-a3fd-3407fec157d1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:43.677541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1340ca-1e13-430f-821c-e8407644d972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:43.678330 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1340ca-1e13-430f-821c-e8407644d972 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:43.678617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1340ca-1e13-430f-821c-e8407644d972 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:43.678898 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1340ca-1e13-430f-821c-e8407644d972 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:43.683130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-217eb9de-c90a-45b2-a3fd-3407fec157d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:43.683130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:43.683130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:43.683130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:43.683130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:43.683130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:43.683130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:43.683130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:43.683130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:43.683130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:43.683130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:43.683130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:43.683130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:55:43.683130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:55:43.683130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:55:43.684020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:55:43.684020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:43.684020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:43.684020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:43.684020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:43.684020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:43.684020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:43.684020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:43.684020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:43.684020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:43.684020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:43.684020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:43.684020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:43.684020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:43.684020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:43.684020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:43.684020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:43.684903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:55:43.684903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:55:43.684903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:55:43.684903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:55:43.684903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:55:43.684903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:55:43.684903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:43.684903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:43.684903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 262/518] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:43.685370 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217eb9de-c90a-45b2-a3fd-3407fec157d1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:43.690365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1340ca-1e13-430f-821c-e8407644d972 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:43.691913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1340ca-1e13-430f-821c-e8407644d972 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:43.709190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-7a1340ca-1e13-430f-821c-e8407644d972 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:43.709190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:43.709190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:43.709190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:43.709190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:43.709190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:43.709190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:43.709190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:43.709190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:43.709190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:43.709190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:43.709190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:43.709190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:43.709190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:43.709190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:43.710021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:43.710021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:43.710021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:43.710021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:43.710021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:43.710021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:43.710021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:43.710021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:43.710021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:43.710021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:43.710021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:43.710021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:43.710021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:43.710021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:43.710021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:43.710021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:43.710021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:43.710886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:43.710886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:43.710886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:43.710886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:43.710886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 257/519] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:43.711140 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1340ca-1e13-430f-821c-e8407644d972 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:43.733808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:43.788468 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217eb9de-c90a-45b2-a3fd-3407fec157d1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:43.801996 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:43.812358 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1340ca-1e13-430f-821c-e8407644d972 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:43.822691 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881f4d2-646d-4db6-be0f-9d7656155e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:43.823340 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881f4d2-646d-4db6-be0f-9d7656155e3f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:43.823625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881f4d2-646d-4db6-be0f-9d7656155e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:43.823906 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881f4d2-646d-4db6-be0f-9d7656155e3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:43.846060 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b075b0-946b-4812-a9a7-f485ef8bac27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:43.846723 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b075b0-946b-4812-a9a7-f485ef8bac27 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:43.847010 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b075b0-946b-4812-a9a7-f485ef8bac27 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:43.847290 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b075b0-946b-4812-a9a7-f485ef8bac27 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:43.848042 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b075b0-946b-4812-a9a7-f485ef8bac27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:43.848376 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b075b0-946b-4812-a9a7-f485ef8bac27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:43.849741 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b075b0-946b-4812-a9a7-f485ef8bac27 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:43.861377 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 258/520] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:43.862140 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b075b0-946b-4812-a9a7-f485ef8bac27 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:43.891920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881f4d2-646d-4db6-be0f-9d7656155e3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:43.893413 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881f4d2-646d-4db6-be0f-9d7656155e3f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:43.942993 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:43.964570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b075b0-946b-4812-a9a7-f485ef8bac27 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:43.997913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8848b-d137-45d5-8795-635d902234a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:43.998584 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8848b-d137-45d5-8795-635d902234a2 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:43.998880 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8848b-d137-45d5-8795-635d902234a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:43.999162 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8848b-d137-45d5-8795-635d902234a2 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:43.999865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa8848b-d137-45d5-8795-635d902234a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:44.001338 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa8848b-d137-45d5-8795-635d902234a2 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:44.002570 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 263/521] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:44.003493 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881f4d2-646d-4db6-be0f-9d7656155e3f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:44.020517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:44.021195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 259/522] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Jul 18 13:55:43 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:55:44.022007 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8848b-d137-45d5-8795-635d902234a2 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:44.117467 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881f4d2-646d-4db6-be0f-9d7656155e3f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:44.136206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8848b-d137-45d5-8795-635d902234a2 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:44.154631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da848c4e-0f3c-4227-a189-4e1dc688e582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:44.155313 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da848c4e-0f3c-4227-a189-4e1dc688e582 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:44.155601 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da848c4e-0f3c-4227-a189-4e1dc688e582 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:44.155881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da848c4e-0f3c-4227-a189-4e1dc688e582 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:44.156770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da848c4e-0f3c-4227-a189-4e1dc688e582 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:44.157087 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da848c4e-0f3c-4227-a189-4e1dc688e582 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:44.158580 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da848c4e-0f3c-4227-a189-4e1dc688e582 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:44.172858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ccb885-9f21-42ad-8d94-e7de0b5e51fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:44.173533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccb885-9f21-42ad-8d94-e7de0b5e51fc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:44.173817 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccb885-9f21-42ad-8d94-e7de0b5e51fc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:44.174098 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccb885-9f21-42ad-8d94-e7de0b5e51fc None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:44.174826 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ccb885-9f21-42ad-8d94-e7de0b5e51fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:44.175793 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ccb885-9f21-42ad-8d94-e7de0b5e51fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:44.177373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ccb885-9f21-42ad-8d94-e7de0b5e51fc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:44.178586 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 264/523] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:55:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:44.179179 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:44.179291 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da848c4e-0f3c-4227-a189-4e1dc688e582 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:44.196684 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 260/524] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:44.197527 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ccb885-9f21-42ad-8d94-e7de0b5e51fc None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:44.270219 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:44.297456 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da848c4e-0f3c-4227-a189-4e1dc688e582 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:44.309238 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ccb885-9f21-42ad-8d94-e7de0b5e51fc None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:44.333623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1400c17-4f24-41bc-9078-29620aa783e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:44.334533 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1400c17-4f24-41bc-9078-29620aa783e2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:44.334821 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1400c17-4f24-41bc-9078-29620aa783e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:44.335238 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1400c17-4f24-41bc-9078-29620aa783e2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:44.342977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a8a63-9558-418f-8fc8-f57a443a87ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:44.343557 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1400c17-4f24-41bc-9078-29620aa783e2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:44.343796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a8a63-9558-418f-8fc8-f57a443a87ac None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:44.344088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a8a63-9558-418f-8fc8-f57a443a87ac None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:44.344370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a8a63-9558-418f-8fc8-f57a443a87ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:44.345039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1400c17-4f24-41bc-9078-29620aa783e2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:44.355031 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a8a63-9558-418f-8fc8-f57a443a87ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:44.356518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a8a63-9558-418f-8fc8-f57a443a87ac None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:44.360747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f1400c17-4f24-41bc-9078-29620aa783e2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 13:55:44.360747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:44.360747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:44.360747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:44.360747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:44.360747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:44.360747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:44.360747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:44.360747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:44.360747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:44.360747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:44.360747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:44.360747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:55:44.360747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:55:44.360747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:55:44.360747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:55:44.361635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:44.361635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:44.361635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:55:44.361635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:44.361635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:55:44.361635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:44.361635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:44.361635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:44.361635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:44.361635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:44.361635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:44.361635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:44.361635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:44.361635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:44.361635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:44.361635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:44.361635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:44.361635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:44.362565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:44.362565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:44.362565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:55:44.362565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:55:44.362565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:55:44.362565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:55:44.362565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:55:44.362565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:55:44.362565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 13:55:44.362565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:44.362565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 265/525] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:55:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:44.363121 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1400c17-4f24-41bc-9078-29620aa783e2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:44.378639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-6e5a8a63-9558-418f-8fc8-f57a443a87ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:44.378639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:44.378639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:44.378639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:44.378639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:44.378639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:44.378639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:44.378639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:44.378639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:44.378639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:44.378639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:44.378639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:44.378639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:44.378639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:44.378639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:44.379547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:44.379547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:44.379547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:44.379547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:44.379547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:44.379547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:44.379547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:44.379547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:44.379547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:44.379547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:44.379547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:44.379547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:44.379547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:44.379547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:44.379547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:44.379547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:44.379547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:44.380511 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:44.380511 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:44.380511 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:44.380511 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:44.380511 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 261/526] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:44.380803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a8a63-9558-418f-8fc8-f57a443a87ac None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:44.431656 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:44.471199 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1400c17-4f24-41bc-9078-29620aa783e2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:44.485326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a8a63-9558-418f-8fc8-f57a443a87ac None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:44.507203 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32b6be-904a-4b28-bb82-5ae7f445b5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:44.508230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32b6be-904a-4b28-bb82-5ae7f445b5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:44.508692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32b6be-904a-4b28-bb82-5ae7f445b5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:44.509114 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32b6be-904a-4b28-bb82-5ae7f445b5e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:44.510086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a32b6be-904a-4b28-bb82-5ae7f445b5e8 None admin] RBAC: Adding query filter params (role.id=5c4ba827ac854bc69cc26c3bcd27dc5c, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=ac717fcd9fc1499db12cf852d45572a8) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:44.510563 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a32b6be-904a-4b28-bb82-5ae7f445b5e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:44.512147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a32b6be-904a-4b28-bb82-5ae7f445b5e8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:44.514160 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:44.520495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab6131-6bd1-441c-955d-0f447fda0a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:44.520966 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 266/527] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:55:44 2022] GET /identity/v3/role_assignments?user.id=ac717fcd9fc1499db12cf852d45572a8&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=5c4ba827ac854bc69cc26c3bcd27dc5c => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:44.521299 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab6131-6bd1-441c-955d-0f447fda0a2b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:44.521594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab6131-6bd1-441c-955d-0f447fda0a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:44.521878 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab6131-6bd1-441c-955d-0f447fda0a2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:44.522361 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32b6be-904a-4b28-bb82-5ae7f445b5e8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:44.535657 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab6131-6bd1-441c-955d-0f447fda0a2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:44.537196 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab6131-6bd1-441c-955d-0f447fda0a2b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:44.553475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-52ab6131-6bd1-441c-955d-0f447fda0a2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:44.553475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:44.553475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:44.553475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:44.553475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:44.553475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:44.553475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:44.553475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:44.553475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:44.553475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:44.553475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:44.553475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:44.553475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:55:44.553475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:55:44.553475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:55:44.554571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:55:44.554571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:44.554571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:44.554571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:44.554571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:44.554571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:44.554571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:44.554571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:44.554571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:44.554571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:44.554571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:44.554571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:44.554571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:44.554571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:44.554571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:44.554571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:44.554571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:44.555696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:55:44.555696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:55:44.555696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:55:44.555696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:55:44.555696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:55:44.555696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:55:44.555696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:44.555696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:44.559610 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 262/528] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:44.560635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab6131-6bd1-441c-955d-0f447fda0a2b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:44.648327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32b6be-904a-4b28-bb82-5ae7f445b5e8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:44.673947 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab6131-6bd1-441c-955d-0f447fda0a2b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:44.685091 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:44.697152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e8892-508a-442e-a01c-8705a9241fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:44.697849 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e8892-508a-442e-a01c-8705a9241fdb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:44.698138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e8892-508a-442e-a01c-8705a9241fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:44.698417 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e8892-508a-442e-a01c-8705a9241fdb None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:44.699844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e8892-508a-442e-a01c-8705a9241fdb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:44.700296 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e8892-508a-442e-a01c-8705a9241fdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:44.702074 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e8892-508a-442e-a01c-8705a9241fdb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:44.716980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76d10a-48a2-443f-969e-b0abf7551f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:44.717891 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76d10a-48a2-443f-969e-b0abf7551f4e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:44.718340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76d10a-48a2-443f-969e-b0abf7551f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:44.718758 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76d10a-48a2-443f-969e-b0abf7551f4e None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:44.719656 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb76d10a-48a2-443f-969e-b0abf7551f4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:44.720609 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb76d10a-48a2-443f-969e-b0abf7551f4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:44.722227 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb76d10a-48a2-443f-969e-b0abf7551f4e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:44.723747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 267/529] 10.222.0.94 () {56 vars in 1139 bytes} [Mon Jul 18 13:55:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:44.724729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e8892-508a-442e-a01c-8705a9241fdb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:44.736655 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 263/530] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:44.737769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76d10a-48a2-443f-969e-b0abf7551f4e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:44.789112 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:44.845824 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76d10a-48a2-443f-969e-b0abf7551f4e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:44.853529 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e8892-508a-442e-a01c-8705a9241fdb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:44.886707 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e913904-2a49-48af-8ef4-ef186981a8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:44.887371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e913904-2a49-48af-8ef4-ef186981a8a4 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:44.887652 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e913904-2a49-48af-8ef4-ef186981a8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:44.888040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e913904-2a49-48af-8ef4-ef186981a8a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:44.907354 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8a823-7660-4d8e-9050-0fd1687d9a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:44.908618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8a823-7660-4d8e-9050-0fd1687d9a6e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:44.909093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8a823-7660-4d8e-9050-0fd1687d9a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:44.909510 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8a823-7660-4d8e-9050-0fd1687d9a6e None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:44.910512 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8a823-7660-4d8e-9050-0fd1687d9a6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:44.911477 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8a823-7660-4d8e-9050-0fd1687d9a6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:44.913671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8a823-7660-4d8e-9050-0fd1687d9a6e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:44.925499 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:44.933002 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 268/531] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:55:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:44.933997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8a823-7660-4d8e-9050-0fd1687d9a6e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:44.973177 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e913904-2a49-48af-8ef4-ef186981a8a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:44.975748 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e913904-2a49-48af-8ef4-ef186981a8a4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:45.054795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8a823-7660-4d8e-9050-0fd1687d9a6e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:45.071739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:45.095502 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8a682-c00e-4e18-adb4-acd990c33c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:45.096591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8a682-c00e-4e18-adb4-acd990c33c81 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:45.097287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8a682-c00e-4e18-adb4-acd990c33c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:45.097578 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8a682-c00e-4e18-adb4-acd990c33c81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:45.101793 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 264/532] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:45.102844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e913904-2a49-48af-8ef4-ef186981a8a4 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:45.105823 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8a682-c00e-4e18-adb4-acd990c33c81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:45.108252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8a682-c00e-4e18-adb4-acd990c33c81 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:45.127591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e2b8a682-c00e-4e18-adb4-acd990c33c81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:45.127591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:45.127591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:45.127591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:45.127591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:45.127591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:45.127591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:45.127591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:45.127591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:45.127591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:45.127591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:45.127591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:45.127591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:55:45.127591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:55:45.127591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:55:45.128689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:55:45.128689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:45.128689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:45.128689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:45.128689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:45.128689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:45.128689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:45.128689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:45.128689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:45.128689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:45.128689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:45.128689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:45.128689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:45.128689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:45.128689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:45.128689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:45.128689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:45.129607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:55:45.129607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:55:45.129607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:55:45.129607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:55:45.129607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:55:45.129607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:55:45.129607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:45.129607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:45.129607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 269/533] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:45.129607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8a682-c00e-4e18-adb4-acd990c33c81 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:45.185801 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:45.233217 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e913904-2a49-48af-8ef4-ef186981a8a4 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:45.269872 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8a682-c00e-4e18-adb4-acd990c33c81 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:45.279075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da16974-e95e-4647-adde-a413eda4759e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:45.279886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da16974-e95e-4647-adde-a413eda4759e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:45.280305 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da16974-e95e-4647-adde-a413eda4759e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:45.280688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da16974-e95e-4647-adde-a413eda4759e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:45.282209 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da16974-e95e-4647-adde-a413eda4759e None admin] RBAC: Adding query filter params (role.id=5c4ba827ac854bc69cc26c3bcd27dc5c, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=503a8228d4754c309fc3f365f4cfc61b) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:45.282652 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da16974-e95e-4647-adde-a413eda4759e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:45.284690 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da16974-e95e-4647-adde-a413eda4759e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:45.300087 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 265/534] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:55:45 2022] GET /identity/v3/role_assignments?user.id=503a8228d4754c309fc3f365f4cfc61b&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=5c4ba827ac854bc69cc26c3bcd27dc5c => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:45.300087 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da16974-e95e-4647-adde-a413eda4759e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:45.335896 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c636eed-120d-41b8-b444-a53cb19ba2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:45.336815 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c636eed-120d-41b8-b444-a53cb19ba2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:45.337107 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c636eed-120d-41b8-b444-a53cb19ba2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:45.337392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c636eed-120d-41b8-b444-a53cb19ba2e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:45.360186 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c636eed-120d-41b8-b444-a53cb19ba2e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:45.361691 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c636eed-120d-41b8-b444-a53cb19ba2e9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:45.398144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-0c636eed-120d-41b8-b444-a53cb19ba2e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:45.398144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:45.398144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:45.398144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:45.398144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:45.398144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:45.398144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:45.398144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:45.398144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:45.398144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:45.398144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:45.398144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:45.398144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:45.398144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:45.398144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:45.399127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:45.399127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:45.399127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:45.399127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:45.399127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:45.399127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:45.399127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:45.399127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:45.399127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:45.399127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:45.399127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:45.399127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:45.399127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:45.399127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:45.399127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:45.399127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:45.399127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:45.399993 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:45.399993 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:45.399993 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:45.399993 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:45.400764 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 270/535] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:45.407248 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c636eed-120d-41b8-b444-a53cb19ba2e9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:45.461497 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:45.498170 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da16974-e95e-4647-adde-a413eda4759e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:45.543347 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0452d6c-ba06-44f8-8e2e-0f9478450555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:45.544083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0452d6c-ba06-44f8-8e2e-0f9478450555 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:45.544705 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:45.545547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0452d6c-ba06-44f8-8e2e-0f9478450555 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:45.545657 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0452d6c-ba06-44f8-8e2e-0f9478450555 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:45.546955 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0452d6c-ba06-44f8-8e2e-0f9478450555 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:45.547593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0452d6c-ba06-44f8-8e2e-0f9478450555 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:45.549273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0452d6c-ba06-44f8-8e2e-0f9478450555 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:45.569871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 266/536] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:55:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:45.576290 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0452d6c-ba06-44f8-8e2e-0f9478450555 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:45.592845 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c636eed-120d-41b8-b444-a53cb19ba2e9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:45.667695 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea6c4ff-caa6-4150-9def-6267ac2f96f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:45.668559 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6c4ff-caa6-4150-9def-6267ac2f96f4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:45.668860 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6c4ff-caa6-4150-9def-6267ac2f96f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:45.669142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6c4ff-caa6-4150-9def-6267ac2f96f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:45.669876 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6c4ff-caa6-4150-9def-6267ac2f96f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:45.670188 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6c4ff-caa6-4150-9def-6267ac2f96f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:45.671567 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6c4ff-caa6-4150-9def-6267ac2f96f4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:45.685420 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 271/537] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:45.696524 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea6c4ff-caa6-4150-9def-6267ac2f96f4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:45.754471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0452d6c-ba06-44f8-8e2e-0f9478450555 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:45.798933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:45.813340 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea6c4ff-caa6-4150-9def-6267ac2f96f4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:45.852298 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c5068-aff1-4254-8d9d-d9134b6cbf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:45.853439 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c5068-aff1-4254-8d9d-d9134b6cbf7c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:45.853740 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c5068-aff1-4254-8d9d-d9134b6cbf7c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:45.854019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c5068-aff1-4254-8d9d-d9134b6cbf7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:45.861099 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c5068-aff1-4254-8d9d-d9134b6cbf7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:45.862781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c5068-aff1-4254-8d9d-d9134b6cbf7c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:45.879449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-210c5068-aff1-4254-8d9d-d9134b6cbf7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:45.879449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:45.879449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:45.879449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:45.879449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:45.879449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:45.879449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:45.879449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:45.879449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:45.879449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:45.879449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:45.879449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:45.879449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:55:45.879449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:55:45.879449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:55:45.880474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:55:45.880474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:45.880474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:45.880474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:45.880474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:45.880474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:45.880474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:45.880474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:45.880474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:45.880474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:45.880474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:45.880474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:45.880474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:45.880474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:45.880474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:45.880474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:45.880474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:45.881513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:55:45.881513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:55:45.881513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:55:45.881513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:55:45.881513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:55:45.881513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:55:45.881513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:45.881513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:45.882031 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc1605-4e76-46f4-ad61-4e28f3171fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:45.882134 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 272/538] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:45.882220 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc1605-4e76-46f4-ad61-4e28f3171fde None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:45.882220 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc1605-4e76-46f4-ad61-4e28f3171fde None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:45.882375 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc1605-4e76-46f4-ad61-4e28f3171fde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:45.883202 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc1605-4e76-46f4-ad61-4e28f3171fde None admin] RBAC: Adding query filter params (role.id=5c4ba827ac854bc69cc26c3bcd27dc5c, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=45ee15f7716541a1a61443069d5cf4f8) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:45.883524 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc1605-4e76-46f4-ad61-4e28f3171fde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:45.887975 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc1605-4e76-46f4-ad61-4e28f3171fde None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:45.891328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c5068-aff1-4254-8d9d-d9134b6cbf7c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:45.896861 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 267/539] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:55:45 2022] GET /identity/v3/role_assignments?user.id=45ee15f7716541a1a61443069d5cf4f8&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=5c4ba827ac854bc69cc26c3bcd27dc5c => generated 259 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:46.072923 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c5068-aff1-4254-8d9d-d9134b6cbf7c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:46.120701 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e5882-c416-48da-a1f8-0ee070d5ef16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:46.121382 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e5882-c416-48da-a1f8-0ee070d5ef16 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:46.121577 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b732c-6264-447e-b506-e669b2a32bf4 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:46.121679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e5882-c416-48da-a1f8-0ee070d5ef16 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:46.121828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b732c-6264-447e-b506-e669b2a32bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:46.121926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e5882-c416-48da-a1f8-0ee070d5ef16 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:46.122088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b732c-6264-447e-b506-e669b2a32bf4 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:46.122800 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e5882-c416-48da-a1f8-0ee070d5ef16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:46.123126 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e5882-c416-48da-a1f8-0ee070d5ef16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:46.123555 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 268/540] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:46.124691 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e5882-c416-48da-a1f8-0ee070d5ef16 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:46.129627 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc0a51-2bbe-4704-ae4e-33eee1b94d97 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:46.129927 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc0a51-2bbe-4704-ae4e-33eee1b94d97 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:46.130209 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc0a51-2bbe-4704-ae4e-33eee1b94d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:46.143094 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 273/541] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:55:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:46.148474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e5882-c416-48da-a1f8-0ee070d5ef16 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:46.155439 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-9ccc0a51-2bbe-4704-ae4e-33eee1b94d97 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:46.157462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9ccc0a51-2bbe-4704-ae4e-33eee1b94d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:46.167084 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:46.225411 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:46.252285 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 269/542] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 13:55:46.290830 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc0a51-2bbe-4704-ae4e-33eee1b94d97 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:46.328452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e5882-c416-48da-a1f8-0ee070d5ef16 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:46.396610 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0dd3d6-1f4b-43f6-8a3f-50cf6b729f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:46.397273 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0dd3d6-1f4b-43f6-8a3f-50cf6b729f78 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:46.397565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0dd3d6-1f4b-43f6-8a3f-50cf6b729f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:46.397846 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0dd3d6-1f4b-43f6-8a3f-50cf6b729f78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:46.399224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0dd3d6-1f4b-43f6-8a3f-50cf6b729f78 None admin] RBAC: Adding query filter params (role.id=5c4ba827ac854bc69cc26c3bcd27dc5c, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=14027a0e13fd4272bd7803e3131b6348) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:46.399542 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0dd3d6-1f4b-43f6-8a3f-50cf6b729f78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:46.401315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0dd3d6-1f4b-43f6-8a3f-50cf6b729f78 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:46.407792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 274/543] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:55:46 2022] GET /identity/v3/role_assignments?user.id=14027a0e13fd4272bd7803e3131b6348&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=5c4ba827ac854bc69cc26c3bcd27dc5c => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:46.486429 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc0a51-2bbe-4704-ae4e-33eee1b94d97 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:46.535717 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649411c-43ca-4802-aa3a-93a04b348f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:46.536567 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649411c-43ca-4802-aa3a-93a04b348f39 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:46.537027 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649411c-43ca-4802-aa3a-93a04b348f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:46.537300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649411c-43ca-4802-aa3a-93a04b348f39 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:46.556255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5649411c-43ca-4802-aa3a-93a04b348f39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:46.564128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-5649411c-43ca-4802-aa3a-93a04b348f39 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:46.564128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:46.564128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:46.564128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:46.564128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:46.564128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:46.564128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:46.564128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:46.564128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:46.564128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:46.564128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:46.564128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:46.564128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:46.564128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:46.564128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:46.564128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:46.565011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:46.565011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:46.565011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:46.565011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:46.565011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:46.565011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:46.565011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:46.565011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:46.565011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:46.565011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:46.565011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:46.565011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:46.565011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:46.565011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:46.565011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:46.565011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:46.565810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:46.565810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:46.565810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:46.565810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:46.565810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:46.565810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:46.565810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:46.565810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:46.565810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:46.565810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:46.565810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:46.565810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:46.565810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 270/544] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:55:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:55:46.575427 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0dd3d6-1f4b-43f6-8a3f-50cf6b729f78 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:46.670623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:46.703748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0dd3d6-1f4b-43f6-8a3f-50cf6b729f78 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:46.741363 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff2073-d545-426e-8be5-38b6b2faadfb None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:46.741670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff2073-d545-426e-8be5-38b6b2faadfb None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:46.741941 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff2073-d545-426e-8be5-38b6b2faadfb None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:46.744003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 271/545] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:46.753319 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16d367-ad44-4778-8fbd-b09282b2c1f5 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:46.753615 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16d367-ad44-4778-8fbd-b09282b2c1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:46.753904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16d367-ad44-4778-8fbd-b09282b2c1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:46.754836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d95e57-8818-470b-ae02-285fa03018d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:46.755518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d95e57-8818-470b-ae02-285fa03018d2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:46.755809 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d95e57-8818-470b-ae02-285fa03018d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:46.756130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d95e57-8818-470b-ae02-285fa03018d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:46.756893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d95e57-8818-470b-ae02-285fa03018d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:46.757207 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d95e57-8818-470b-ae02-285fa03018d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:46.758739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d95e57-8818-470b-ae02-285fa03018d2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:46.778204 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 275/546] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Jul 18 13:55:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:46.783826 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d95e57-8818-470b-ae02-285fa03018d2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:46.785546 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-2a16d367-ad44-4778-8fbd-b09282b2c1f5 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:46.786792 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2a16d367-ad44-4778-8fbd-b09282b2c1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:46.857494 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:46.907908 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 272/547] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:46.931345 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16d367-ad44-4778-8fbd-b09282b2c1f5 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:46.983986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d95e57-8818-470b-ae02-285fa03018d2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:47.054723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e79ce3-5c46-4f83-9407-dacd88233eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:47.056163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e79ce3-5c46-4f83-9407-dacd88233eaf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:47.056605 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e79ce3-5c46-4f83-9407-dacd88233eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:47.056892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e79ce3-5c46-4f83-9407-dacd88233eaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:47.058773 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:47.069974 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e79ce3-5c46-4f83-9407-dacd88233eaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:47.071462 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e79ce3-5c46-4f83-9407-dacd88233eaf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:47.091626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-d2e79ce3-5c46-4f83-9407-dacd88233eaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:47.091626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:47.091626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:47.091626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:47.091626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:47.091626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:47.091626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:47.091626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:47.091626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:47.091626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:47.091626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:47.091626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:47.091626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:47.091626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:47.091626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:47.092582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:47.092582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:47.092582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:47.092582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:47.092582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:47.092582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:47.092582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:47.092582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:47.092582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:47.092582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:47.092582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:47.092582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:47.092582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:47.092582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:47.092582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:47.092582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:47.092582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:47.093884 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:47.093884 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:47.093884 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:47.093884 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:47.093884 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 276/548] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:55:47.101330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e79ce3-5c46-4f83-9407-dacd88233eaf None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:47.139669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16d367-ad44-4778-8fbd-b09282b2c1f5 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:47.208488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be7f78-f099-4050-8a38-781fbf9c5dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:47.209483 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be7f78-f099-4050-8a38-781fbf9c5dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:47.209771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be7f78-f099-4050-8a38-781fbf9c5dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:47.210054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be7f78-f099-4050-8a38-781fbf9c5dd6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:47.233569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be7f78-f099-4050-8a38-781fbf9c5dd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:47.241704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-f0be7f78-f099-4050-8a38-781fbf9c5dd6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:47.241704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:47.241704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:47.241704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:47.241704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:47.241704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:47.241704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:47.241704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:47.241704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:47.241704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:47.241704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:47.241704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:47.241704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:47.241704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:47.241704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:47.241704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:47.242813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:47.242813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:47.242813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:47.242813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:47.242813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:47.242813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:47.242813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:47.242813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:47.242813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:47.242813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:47.242813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:47.242813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:47.242813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:47.242813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:47.242813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:47.242813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:47.244622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:47.244622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:47.244622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:47.244622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:47.244622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:47.244622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:47.244622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:47.244622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:47.244622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:47.244622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:47.244622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:47.244622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:47.244622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 273/549] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:55:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:47.249410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be7f78-f099-4050-8a38-781fbf9c5dd6 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:47.283198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e79ce3-5c46-4f83-9407-dacd88233eaf None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:47.309833 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:47.329082 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0252996-357f-4e8d-a14b-3491d2fed379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:47.331648 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0252996-357f-4e8d-a14b-3491d2fed379 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:47.332168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0252996-357f-4e8d-a14b-3491d2fed379 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:47.333635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0252996-357f-4e8d-a14b-3491d2fed379 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:47.334580 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0252996-357f-4e8d-a14b-3491d2fed379 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:47.336966 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0252996-357f-4e8d-a14b-3491d2fed379 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:47.338622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0252996-357f-4e8d-a14b-3491d2fed379 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:47.366461 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 277/550] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:47.368897 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0252996-357f-4e8d-a14b-3491d2fed379 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:47.442810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be7f78-f099-4050-8a38-781fbf9c5dd6 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:47.469276 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:47.514423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3523e-5675-4e7f-9444-faae380df820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:47.516816 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3523e-5675-4e7f-9444-faae380df820 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:47.517912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3523e-5675-4e7f-9444-faae380df820 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:47.518478 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3523e-5675-4e7f-9444-faae380df820 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:47.519586 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3523e-5675-4e7f-9444-faae380df820 None admin] RBAC: Adding query filter params (name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:47.524400 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3523e-5675-4e7f-9444-faae380df820 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:47.526032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3523e-5675-4e7f-9444-faae380df820 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:47.541684 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 274/551] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Jul 18 13:55:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:55:47.547899 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3523e-5675-4e7f-9444-faae380df820 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:47.577465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0252996-357f-4e8d-a14b-3491d2fed379 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:47.644039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866f201-6d4f-4339-9d38-6ce96a0db35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:47.648166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866f201-6d4f-4339-9d38-6ce96a0db35a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:47.649259 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866f201-6d4f-4339-9d38-6ce96a0db35a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:47.649752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866f201-6d4f-4339-9d38-6ce96a0db35a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:47.663752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866f201-6d4f-4339-9d38-6ce96a0db35a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:47.666452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866f201-6d4f-4339-9d38-6ce96a0db35a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:47.696713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5866f201-6d4f-4339-9d38-6ce96a0db35a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 13:55:47.696713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:47.696713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:47.696713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:47.696713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:47.696713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:47.696713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:47.696713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:47.696713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:47.696713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:47.696713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:47.696713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:47.696713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:55:47.696713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:55:47.696713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:55:47.696713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:55:47.698183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:47.698183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:47.698183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:55:47.698183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:47.698183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:55:47.698183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:47.698183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:47.698183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:47.698183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:47.698183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:47.698183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:47.698183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:47.698183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:47.698183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:47.698183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:47.698183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:47.698183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:47.698183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:47.699737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:47.699737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:47.699737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:55:47.699737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:55:47.699737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:55:47.699737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:55:47.699737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:55:47.699737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:55:47.699737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 13:55:47.699737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:47.699737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 278/552] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:55:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:55:47.704108 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866f201-6d4f-4339-9d38-6ce96a0db35a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:47.728965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:47.736167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3523e-5675-4e7f-9444-faae380df820 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:47.807095 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc58de-2529-4eb6-b988-dbe42059c678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:47.808361 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc58de-2529-4eb6-b988-dbe42059c678 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:47.810086 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc58de-2529-4eb6-b988-dbe42059c678 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:47.810637 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc58de-2529-4eb6-b988-dbe42059c678 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:47.828765 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afc58de-2529-4eb6-b988-dbe42059c678 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:47.831181 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afc58de-2529-4eb6-b988-dbe42059c678 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:47.858518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-6afc58de-2529-4eb6-b988-dbe42059c678 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:47.858518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:47.858518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:47.858518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:47.858518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:47.858518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:47.858518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:47.858518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:47.858518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:47.858518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:47.858518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:47.858518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:47.858518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:47.858518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:47.858518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:47.859455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:47.859455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:47.859455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:47.859455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:47.859455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:47.859455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:47.859455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:47.859455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:47.859455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:47.859455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:47.859455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:47.859455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:47.859455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:47.859455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:47.859455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:47.859455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:47.859455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:47.861292 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:47.861292 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:47.861292 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:47.861292 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:47.864577 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 275/553] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:47.866141 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f56c1-2533-42c4-a1d9-8c3d1a232382 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:47.866644 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f56c1-2533-42c4-a1d9-8c3d1a232382 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:47.867133 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f56c1-2533-42c4-a1d9-8c3d1a232382 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:47.869182 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 276/554] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:47.870173 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f56c1-2533-42c4-a1d9-8c3d1a232382 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:47.894726 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866f201-6d4f-4339-9d38-6ce96a0db35a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:47.905357 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:47.939261 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43fdb1c-4e09-4f94-9641-61cbd83daaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:47.939953 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fdb1c-4e09-4f94-9641-61cbd83daaaf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:47.940256 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fdb1c-4e09-4f94-9641-61cbd83daaaf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:47.940533 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fdb1c-4e09-4f94-9641-61cbd83daaaf None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:47.941397 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43fdb1c-4e09-4f94-9641-61cbd83daaaf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:47.941696 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43fdb1c-4e09-4f94-9641-61cbd83daaaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:47.943130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43fdb1c-4e09-4f94-9641-61cbd83daaaf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:47.968018 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 279/555] 10.222.0.94 () {56 vars in 1135 bytes} [Mon Jul 18 13:55:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:47.969629 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc792a-0ce9-4ee9-814f-b085d20491e4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:47.970135 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc792a-0ce9-4ee9-814f-b085d20491e4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:47.970797 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc792a-0ce9-4ee9-814f-b085d20491e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:47.987640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bbcc792a-0ce9-4ee9-814f-b085d20491e4 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:47.989247 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bbcc792a-0ce9-4ee9-814f-b085d20491e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:48.015326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f56c1-2533-42c4-a1d9-8c3d1a232382 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:48.051930 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e9c7a-c677-4435-ac31-07ebb3fd1d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:48.052634 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e9c7a-c677-4435-ac31-07ebb3fd1d12 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:48.052919 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e9c7a-c677-4435-ac31-07ebb3fd1d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:48.053193 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e9c7a-c677-4435-ac31-07ebb3fd1d12 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:48.054103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e9c7a-c677-4435-ac31-07ebb3fd1d12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:48.054324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e9c7a-c677-4435-ac31-07ebb3fd1d12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:48.056491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e9c7a-c677-4435-ac31-07ebb3fd1d12 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:48.062939 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 280/556] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:48.064172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc792a-0ce9-4ee9-814f-b085d20491e4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:48.082149 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 277/557] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:48.087770 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e9c7a-c677-4435-ac31-07ebb3fd1d12 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:48.119555 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:48.163670 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:48.171793 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc792a-0ce9-4ee9-814f-b085d20491e4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:48.207863 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e9c7a-c677-4435-ac31-07ebb3fd1d12 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:48.210102 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9384f7b-acd5-4384-aa61-5502604d4dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:48.210761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9384f7b-acd5-4384-aa61-5502604d4dee None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:48.211077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9384f7b-acd5-4384-aa61-5502604d4dee None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:48.211355 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9384f7b-acd5-4384-aa61-5502604d4dee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:48.244953 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595dad19-a2ca-4f42-b502-3b2c6aab9982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:48.245644 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595dad19-a2ca-4f42-b502-3b2c6aab9982 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:48.245931 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595dad19-a2ca-4f42-b502-3b2c6aab9982 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:48.246204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595dad19-a2ca-4f42-b502-3b2c6aab9982 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:48.254138 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595dad19-a2ca-4f42-b502-3b2c6aab9982 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:48.255570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595dad19-a2ca-4f42-b502-3b2c6aab9982 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:48.272106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-595dad19-a2ca-4f42-b502-3b2c6aab9982 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 13:55:48.272106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:48.272106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:48.272106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:48.272106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:48.272106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:48.272106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:48.272106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:48.272106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:48.272106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:48.272106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:48.272106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:48.272106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:55:48.272106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:55:48.272106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:55:48.272106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:55:48.273595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:48.273595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:48.273595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:55:48.273595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:48.273595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:55:48.273595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:48.273595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:48.273595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:48.273595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:48.273595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:48.273595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:48.273595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:48.273595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:48.273595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:48.273595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:48.273595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:48.273595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:48.273595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:48.274582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:48.274582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:48.274582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:55:48.274582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:55:48.274582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:55:48.274582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:55:48.274582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:55:48.274582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:55:48.274582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 13:55:48.274582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:48.274582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 278/558] 10.222.0.94 () {56 vars in 1093 bytes} [Mon Jul 18 13:55:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:55:48.274582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595dad19-a2ca-4f42-b502-3b2c6aab9982 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:48.290253 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9384f7b-acd5-4384-aa61-5502604d4dee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:48.291735 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9384f7b-acd5-4384-aa61-5502604d4dee None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:48.349387 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:48.383881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595dad19-a2ca-4f42-b502-3b2c6aab9982 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:48.389413 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:48.397268 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 281/559] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:48.398696 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a48e3f-77f4-47fd-a445-be7998317dea None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:48.398992 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a48e3f-77f4-47fd-a445-be7998317dea None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:48.399274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a48e3f-77f4-47fd-a445-be7998317dea None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:48.404063 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 282/560] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:48.404063 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a48e3f-77f4-47fd-a445-be7998317dea None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:48.427550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78dae8-6ffd-4e84-859a-d6cbe3bfa1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:48.428611 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78dae8-6ffd-4e84-859a-d6cbe3bfa1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:48.428901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78dae8-6ffd-4e84-859a-d6cbe3bfa1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:48.429181 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78dae8-6ffd-4e84-859a-d6cbe3bfa1d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:48.440555 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78dae8-6ffd-4e84-859a-d6cbe3bfa1d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:48.449333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-3b78dae8-6ffd-4e84-859a-d6cbe3bfa1d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:48.449333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:48.449333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:48.449333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:48.449333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:48.449333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:48.449333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:48.449333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:48.449333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:48.449333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:48.449333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:48.449333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:48.449333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:48.449333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:48.449333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:48.449333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:48.450364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:48.450364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:48.450364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:48.450364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:48.450364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:48.450364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:48.450364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:48.450364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:48.450364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:48.450364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:48.450364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:48.450364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:48.450364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:48.450364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:48.450364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:48.450364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:48.451201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:48.451201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:48.451201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:48.451201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:48.451201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:48.451201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:48.451201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:48.451201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:48.451201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:48.451201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:48.451201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:48.451201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:48.451201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 279/561] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:55:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:48.451904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78dae8-6ffd-4e84-859a-d6cbe3bfa1d1 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:48.514050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a48e3f-77f4-47fd-a445-be7998317dea None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:48.549709 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92976b6-0716-4ecb-a235-7f7f428cd534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:48.550638 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92976b6-0716-4ecb-a235-7f7f428cd534 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:48.550929 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92976b6-0716-4ecb-a235-7f7f428cd534 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:48.551209 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92976b6-0716-4ecb-a235-7f7f428cd534 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:48.552082 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92976b6-0716-4ecb-a235-7f7f428cd534 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:48.552400 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92976b6-0716-4ecb-a235-7f7f428cd534 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:48.553862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92976b6-0716-4ecb-a235-7f7f428cd534 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:48.559630 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78dae8-6ffd-4e84-859a-d6cbe3bfa1d1 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:48.571734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 283/562] 10.222.0.94 () {56 vars in 1133 bytes} [Mon Jul 18 13:55:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:48.573026 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0364555-3acf-47c9-aa93-e307b1938e9a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:48.573318 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0364555-3acf-47c9-aa93-e307b1938e9a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:48.573594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0364555-3acf-47c9-aa93-e307b1938e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:48.592553 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d0364555-3acf-47c9-aa93-e307b1938e9a None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:48.593854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d0364555-3acf-47c9-aa93-e307b1938e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:48.594851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:48.601546 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc1ba4-5d66-4689-b3ed-5e4039378fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:48.602382 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc1ba4-5d66-4689-b3ed-5e4039378fee None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:48.602669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc1ba4-5d66-4689-b3ed-5e4039378fee None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:48.602950 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc1ba4-5d66-4689-b3ed-5e4039378fee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:48.613953 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bc1ba4-5d66-4689-b3ed-5e4039378fee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:48.615606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bc1ba4-5d66-4689-b3ed-5e4039378fee None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:48.632858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-c3bc1ba4-5d66-4689-b3ed-5e4039378fee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:48.632858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:48.632858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:48.632858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:48.632858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:48.632858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:48.632858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:48.632858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:48.632858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:48.632858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:48.632858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:48.632858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:48.632858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:48.632858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:48.632858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:48.634280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:48.634280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:48.634280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:48.634280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:48.634280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:48.634280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:48.634280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:48.634280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:48.634280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:48.634280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:48.634280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:48.634280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:48.634280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:48.634280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:48.634280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:48.634280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:48.634280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:48.635224 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:48.635224 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:48.635224 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:48.635224 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:48.635224 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 280/563] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:48.636170 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc1ba4-5d66-4689-b3ed-5e4039378fee None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:48.650128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:48.671559 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 284/564] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:48.672499 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0364555-3acf-47c9-aa93-e307b1938e9a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:48.757049 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc1ba4-5d66-4689-b3ed-5e4039378fee None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:48.781857 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0364555-3acf-47c9-aa93-e307b1938e9a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:48.790519 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa61fb-8701-4df2-ade3-9d0c82347b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:48.791178 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa61fb-8701-4df2-ade3-9d0c82347b76 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:48.791464 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa61fb-8701-4df2-ade3-9d0c82347b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:48.791742 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa61fb-8701-4df2-ade3-9d0c82347b76 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:48.792515 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa61fb-8701-4df2-ade3-9d0c82347b76 None admin] RBAC: Adding query filter params (name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:48.792824 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa61fb-8701-4df2-ade3-9d0c82347b76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:48.794203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa61fb-8701-4df2-ade3-9d0c82347b76 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:48.803076 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 281/565] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Jul 18 13:55:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:48.803994 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa61fb-8701-4df2-ade3-9d0c82347b76 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:48.815775 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abe86c-1b40-406c-8e6e-5f71a2bbc0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:48.816610 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abe86c-1b40-406c-8e6e-5f71a2bbc0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:48.816905 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abe86c-1b40-406c-8e6e-5f71a2bbc0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:48.817187 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abe86c-1b40-406c-8e6e-5f71a2bbc0f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:48.827913 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abe86c-1b40-406c-8e6e-5f71a2bbc0f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:48.829762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abe86c-1b40-406c-8e6e-5f71a2bbc0f9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:48.847140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-62abe86c-1b40-406c-8e6e-5f71a2bbc0f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:48.847140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:48.847140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:48.847140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:48.847140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:48.847140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:48.847140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:48.847140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:48.847140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:48.847140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:48.847140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:48.847140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:48.847140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:48.847140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:48.847140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:48.848250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:48.848250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:48.848250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:48.848250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:48.848250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:48.848250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:48.848250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:48.848250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:48.848250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:48.848250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:48.848250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:48.848250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:48.848250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:48.848250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:48.848250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:48.848250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:48.848250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:48.849297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:48.849297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:48.849297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:48.849297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:48.849297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 285/566] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:48.849648 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abe86c-1b40-406c-8e6e-5f71a2bbc0f9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:48.854431 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:48.882391 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:48.909582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa61fb-8701-4df2-ade3-9d0c82347b76 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:48.941959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9a6d2-090e-436d-9279-9c2f89a27e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:48.942619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9a6d2-090e-436d-9279-9c2f89a27e5b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:48.942938 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9a6d2-090e-436d-9279-9c2f89a27e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:48.946546 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9a6d2-090e-436d-9279-9c2f89a27e5b None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:48.946546 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9a6d2-090e-436d-9279-9c2f89a27e5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:48.946546 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9a6d2-090e-436d-9279-9c2f89a27e5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:48.946546 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9a6d2-090e-436d-9279-9c2f89a27e5b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:48.953783 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abe86c-1b40-406c-8e6e-5f71a2bbc0f9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:48.959182 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 282/567] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:48.960532 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88deb499-0efe-46f9-b88f-3dec0cd76d61 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:48.960824 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88deb499-0efe-46f9-b88f-3dec0cd76d61 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:48.961154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88deb499-0efe-46f9-b88f-3dec0cd76d61 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:48.963963 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 283/568] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:48.964081 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88deb499-0efe-46f9-b88f-3dec0cd76d61 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:48.988491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eeeaa7-1fb2-4cb1-ae5d-77335bc77ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:48.989044 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eeeaa7-1fb2-4cb1-ae5d-77335bc77ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:48.989333 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eeeaa7-1fb2-4cb1-ae5d-77335bc77ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:48.989609 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eeeaa7-1fb2-4cb1-ae5d-77335bc77ef0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:49.000352 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eeeaa7-1fb2-4cb1-ae5d-77335bc77ef0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:49.007994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b7eeeaa7-1fb2-4cb1-ae5d-77335bc77ef0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:49.007994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:49.007994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:49.007994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:49.007994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:49.007994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:49.007994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:49.007994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:49.007994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:49.007994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:49.007994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:49.007994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:49.007994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:49.007994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:49.007994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:49.007994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:49.009085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:49.009085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:49.009085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:49.009085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:49.009085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:49.009085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:49.009085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:49.009085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:49.009085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:49.009085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:49.009085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:49.009085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:49.009085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:49.009085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:49.009085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:49.009085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:49.009943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:49.009943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:49.009943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:49.009943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:49.009943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:49.009943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:49.009943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:49.009943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:49.009943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:49.009943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:49.009943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:49.009943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:49.009943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 286/569] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:55:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:49.009943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eeeaa7-1fb2-4cb1-ae5d-77335bc77ef0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:49.067669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88deb499-0efe-46f9-b88f-3dec0cd76d61 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:49.069880 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:49.102698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180cf49-6e21-4f80-8b4c-98fa888172b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:49.103489 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180cf49-6e21-4f80-8b4c-98fa888172b1 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:49.103775 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180cf49-6e21-4f80-8b4c-98fa888172b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:49.104116 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180cf49-6e21-4f80-8b4c-98fa888172b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:49.105485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:49.112612 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eeeaa7-1fb2-4cb1-ae5d-77335bc77ef0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:49.114533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5180cf49-6e21-4f80-8b4c-98fa888172b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:49.116080 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5180cf49-6e21-4f80-8b4c-98fa888172b1 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:49.133663 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-5180cf49-6e21-4f80-8b4c-98fa888172b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:49.133663 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:49.133663 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:49.133663 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:49.133663 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:49.133663 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:49.133663 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:49.133663 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:49.133663 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:49.133663 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:49.133663 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:49.133663 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:49.133663 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:49.133663 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:49.133663 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:49.134739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:49.134739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:49.134739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:49.134739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:49.134739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:49.134739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:49.134739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:49.134739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:49.134739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:49.134739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:49.134739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:49.134739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:49.134739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:49.134739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:49.134739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:49.134739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:49.134739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:49.135818 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:49.135818 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:49.135818 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:49.135818 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:49.135818 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 284/570] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:49.135818 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180cf49-6e21-4f80-8b4c-98fa888172b1 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:49.150506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b237d0-4726-4702-9905-61289f9097a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:49.151168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b237d0-4726-4702-9905-61289f9097a3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:49.151453 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b237d0-4726-4702-9905-61289f9097a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:49.151733 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b237d0-4726-4702-9905-61289f9097a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:49.152510 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b237d0-4726-4702-9905-61289f9097a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:49.152819 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b237d0-4726-4702-9905-61289f9097a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:49.154207 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b237d0-4726-4702-9905-61289f9097a3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:49.166255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 287/571] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:49.167475 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebf8e6-c4d8-4a37-b613-1bffcc10a884 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:49.167775 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebf8e6-c4d8-4a37-b613-1bffcc10a884 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:49.168080 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebf8e6-c4d8-4a37-b613-1bffcc10a884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:49.184263 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ddebf8e6-c4d8-4a37-b613-1bffcc10a884 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:49.185466 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ddebf8e6-c4d8-4a37-b613-1bffcc10a884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:49.238464 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180cf49-6e21-4f80-8b4c-98fa888172b1 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:49.250925 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 288/572] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:49.251751 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebf8e6-c4d8-4a37-b613-1bffcc10a884 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:49.280432 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5708de8-96b7-4174-bb5e-0d9409a9de16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:49.281353 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5708de8-96b7-4174-bb5e-0d9409a9de16 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:49.281744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5708de8-96b7-4174-bb5e-0d9409a9de16 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:49.282034 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5708de8-96b7-4174-bb5e-0d9409a9de16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:49.288713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5708de8-96b7-4174-bb5e-0d9409a9de16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:49.290678 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5708de8-96b7-4174-bb5e-0d9409a9de16 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:49.304052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-b5708de8-96b7-4174-bb5e-0d9409a9de16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:49.304052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:49.304052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:49.304052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:49.304052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:49.304052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:49.304052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:49.304052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:49.304052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:49.304052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:49.304052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:49.304052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:49.304052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:55:49.304052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:55:49.304052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:55:49.305681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:55:49.305681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:49.305681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:49.305681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:49.305681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:49.305681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:49.305681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:49.305681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:49.305681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:49.305681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:49.305681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:49.305681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:49.305681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:49.305681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:49.305681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:49.305681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:49.305681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:49.306796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:55:49.306796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:55:49.306796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:55:49.306796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:55:49.306796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:55:49.306796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:55:49.306796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:49.306796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:49.307503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 285/573] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:49.308351 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5708de8-96b7-4174-bb5e-0d9409a9de16 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:49.327234 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:49.335222 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:49.352646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebf8e6-c4d8-4a37-b613-1bffcc10a884 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:49.385699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3087c63-6f28-473f-bcf7-8bfc0b89c79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:49.386362 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3087c63-6f28-473f-bcf7-8bfc0b89c79c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:49.386652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3087c63-6f28-473f-bcf7-8bfc0b89c79c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:49.386933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3087c63-6f28-473f-bcf7-8bfc0b89c79c None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:49.387674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3087c63-6f28-473f-bcf7-8bfc0b89c79c None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:49.388017 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3087c63-6f28-473f-bcf7-8bfc0b89c79c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:49.389383 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3087c63-6f28-473f-bcf7-8bfc0b89c79c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:49.397043 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 289/574] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Jul 18 13:55:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:49.397806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3087c63-6f28-473f-bcf7-8bfc0b89c79c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:49.408945 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5708de8-96b7-4174-bb5e-0d9409a9de16 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:49.442347 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958fbbe-e1e8-4bd8-9de0-d09c28484993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:49.443017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958fbbe-e1e8-4bd8-9de0-d09c28484993 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:49.443303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958fbbe-e1e8-4bd8-9de0-d09c28484993 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:49.443583 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958fbbe-e1e8-4bd8-9de0-d09c28484993 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:49.444366 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958fbbe-e1e8-4bd8-9de0-d09c28484993 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:49.444679 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958fbbe-e1e8-4bd8-9de0-d09c28484993 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:49.446088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958fbbe-e1e8-4bd8-9de0-d09c28484993 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:49.458148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 286/575] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:49.458953 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958fbbe-e1e8-4bd8-9de0-d09c28484993 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:49.500414 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3087c63-6f28-473f-bcf7-8bfc0b89c79c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:49.533880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc88b7a-391a-4b3b-87cc-287584de223e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:49.534663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc88b7a-391a-4b3b-87cc-287584de223e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:49.534948 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc88b7a-391a-4b3b-87cc-287584de223e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:49.535233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc88b7a-391a-4b3b-87cc-287584de223e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:49.537349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:49.543130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc88b7a-391a-4b3b-87cc-287584de223e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:49.544612 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc88b7a-391a-4b3b-87cc-287584de223e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:49.558776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-abc88b7a-391a-4b3b-87cc-287584de223e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:49.558776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:49.558776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:49.558776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:49.558776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:49.558776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:49.558776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:49.558776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:49.558776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:49.558776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:49.558776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:49.558776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:49.558776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:55:49.558776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:55:49.558776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:55:49.559660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:55:49.559660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:49.559660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:49.559660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:49.559660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:49.559660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:49.559660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:49.559660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:49.559660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:49.559660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:49.559660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:49.559660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:49.559660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:49.559660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:49.559660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:49.559660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:49.559660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:49.561140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:55:49.561140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:55:49.561140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:55:49.561140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:55:49.561140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:55:49.561140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:55:49.561140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:49.561140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:49.561140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 290/576] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:49.561661 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958fbbe-e1e8-4bd8-9de0-d09c28484993 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:49.562061 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc88b7a-391a-4b3b-87cc-287584de223e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:49.568889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:49.595645 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba56052-df69-4af1-8e69-2d007126c90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:49.596415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba56052-df69-4af1-8e69-2d007126c90c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:49.596703 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba56052-df69-4af1-8e69-2d007126c90c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:49.596980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba56052-df69-4af1-8e69-2d007126c90c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:49.607661 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba56052-df69-4af1-8e69-2d007126c90c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:49.615354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-eba56052-df69-4af1-8e69-2d007126c90c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:49.615354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:49.615354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:49.615354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:49.615354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:49.615354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:49.615354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:49.615354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:49.615354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:49.615354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:49.615354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:49.615354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:49.615354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:49.615354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:49.615354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:49.615354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:49.616315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:49.616315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:49.616315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:49.616315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:49.616315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:49.616315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:49.616315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:49.616315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:49.616315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:49.616315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:49.616315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:49.616315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:49.616315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:49.616315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:49.616315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:49.616315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:49.617154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:49.617154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:49.617154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:49.617154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:49.617154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:49.617154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:49.617154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:49.617154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:49.617154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:49.617154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:49.617154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:49.617154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:49.617154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 287/577] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:55:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:49.617929 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba56052-df69-4af1-8e69-2d007126c90c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:49.664786 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc88b7a-391a-4b3b-87cc-287584de223e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:49.696953 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd240db-c3f3-4a97-9a2d-4ab5c765517b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:49.697603 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd240db-c3f3-4a97-9a2d-4ab5c765517b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:49.697885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd240db-c3f3-4a97-9a2d-4ab5c765517b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:49.698166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd240db-c3f3-4a97-9a2d-4ab5c765517b None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:49.698948 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd240db-c3f3-4a97-9a2d-4ab5c765517b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:49.699255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd240db-c3f3-4a97-9a2d-4ab5c765517b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:49.700718 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd240db-c3f3-4a97-9a2d-4ab5c765517b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:49.712589 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 291/578] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:55:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:49.713351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd240db-c3f3-4a97-9a2d-4ab5c765517b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:49.715542 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba56052-df69-4af1-8e69-2d007126c90c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:49.748390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532da0ec-4409-45dc-8363-3d035d950837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:49.749175 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532da0ec-4409-45dc-8363-3d035d950837 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:49.749462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532da0ec-4409-45dc-8363-3d035d950837 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:49.749740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532da0ec-4409-45dc-8363-3d035d950837 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:49.752539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:49.763028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532da0ec-4409-45dc-8363-3d035d950837 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:49.765574 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532da0ec-4409-45dc-8363-3d035d950837 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:49.791140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:49.795150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-532da0ec-4409-45dc-8363-3d035d950837 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:49.795150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:49.795150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:49.795150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:49.795150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:49.795150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:49.795150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:49.795150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:49.795150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:49.795150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:49.795150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:49.795150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:49.795150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:49.795150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:49.795150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:49.796212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:49.796212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:49.796212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:49.796212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:49.796212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:49.796212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:49.796212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:49.796212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:49.796212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:49.796212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:49.796212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:49.796212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:49.796212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:49.796212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:49.796212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:49.796212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:49.796212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:49.797205 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:49.797205 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:49.797205 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:49.797205 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:49.797611 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 288/579] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:49.798626 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532da0ec-4409-45dc-8363-3d035d950837 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:49.832610 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd240db-c3f3-4a97-9a2d-4ab5c765517b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:49.864812 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2fb97-b29b-4b30-afcb-0149550509e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:49.865529 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2fb97-b29b-4b30-afcb-0149550509e8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:49.865809 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2fb97-b29b-4b30-afcb-0149550509e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:49.866084 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2fb97-b29b-4b30-afcb-0149550509e8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:49.873353 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2fb97-b29b-4b30-afcb-0149550509e8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:49.874798 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2fb97-b29b-4b30-afcb-0149550509e8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:49.889046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-02b2fb97-b29b-4b30-afcb-0149550509e8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 13:55:49.889046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:49.889046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:49.889046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:49.889046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:49.889046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:49.889046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:49.889046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:49.889046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:49.889046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:49.889046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:49.889046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:49.889046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:55:49.889046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:55:49.889046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:55:49.889046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:55:49.890152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:49.890152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:49.890152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:55:49.890152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:49.890152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:55:49.890152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:49.890152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:49.890152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:49.890152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:49.890152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:49.890152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:49.890152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:49.890152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:49.890152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:49.890152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:49.890152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:49.890152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:49.890152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:49.896105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:49.896105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:49.896105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:55:49.896105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:55:49.896105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:55:49.896105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:55:49.896105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:55:49.896105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:55:49.896105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 13:55:49.896105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:49.896105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 292/580] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:49.896105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2fb97-b29b-4b30-afcb-0149550509e8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:49.897020 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532da0ec-4409-45dc-8363-3d035d950837 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:49.929731 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0aaf21-dcd7-496d-b952-664c104ddc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:49.930383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0aaf21-dcd7-496d-b952-664c104ddc15 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:49.930670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0aaf21-dcd7-496d-b952-664c104ddc15 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:49.930951 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0aaf21-dcd7-496d-b952-664c104ddc15 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:49.931730 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0aaf21-dcd7-496d-b952-664c104ddc15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:49.932075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0aaf21-dcd7-496d-b952-664c104ddc15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:49.933509 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0aaf21-dcd7-496d-b952-664c104ddc15 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:49.946800 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 289/581] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:55:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:49.947917 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0aaf21-dcd7-496d-b952-664c104ddc15 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:49.994130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2fb97-b29b-4b30-afcb-0149550509e8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:49.997709 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:50.020133 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:50.029619 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf72804-1adb-42a6-8b06-6356f47367f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:50.030279 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf72804-1adb-42a6-8b06-6356f47367f6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:50.030567 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf72804-1adb-42a6-8b06-6356f47367f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:50.030848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf72804-1adb-42a6-8b06-6356f47367f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:50.031588 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf72804-1adb-42a6-8b06-6356f47367f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:50.031897 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf72804-1adb-42a6-8b06-6356f47367f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:50.033299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf72804-1adb-42a6-8b06-6356f47367f6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:50.041306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 293/582] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Jul 18 13:55:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:50.042348 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf72804-1adb-42a6-8b06-6356f47367f6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:50.055001 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0aaf21-dcd7-496d-b952-664c104ddc15 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:50.090318 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a9827-e27f-4642-8387-56ceef9bbf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:50.090975 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a9827-e27f-4642-8387-56ceef9bbf71 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:50.091267 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a9827-e27f-4642-8387-56ceef9bbf71 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:50.091552 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a9827-e27f-4642-8387-56ceef9bbf71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:50.092420 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a9827-e27f-4642-8387-56ceef9bbf71 None admin] RBAC: Adding query filter params (role.id=5c4ba827ac854bc69cc26c3bcd27dc5c, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=cbf513ee25374e49b3d8ce28843e393a) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:50.092738 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a9827-e27f-4642-8387-56ceef9bbf71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:50.094176 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a9827-e27f-4642-8387-56ceef9bbf71 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:50.099520 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 290/583] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:55:49 2022] GET /identity/v3/role_assignments?user.id=cbf513ee25374e49b3d8ce28843e393a&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=5c4ba827ac854bc69cc26c3bcd27dc5c => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:50.100315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a9827-e27f-4642-8387-56ceef9bbf71 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:50.142740 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf72804-1adb-42a6-8b06-6356f47367f6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:50.183480 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10b196-e530-4fd9-9a5d-2a00c805d486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:50.184194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10b196-e530-4fd9-9a5d-2a00c805d486 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:50.184488 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10b196-e530-4fd9-9a5d-2a00c805d486 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:50.184778 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10b196-e530-4fd9-9a5d-2a00c805d486 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:50.185535 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd10b196-e530-4fd9-9a5d-2a00c805d486 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:50.185855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd10b196-e530-4fd9-9a5d-2a00c805d486 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:50.187317 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd10b196-e530-4fd9-9a5d-2a00c805d486 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:50.205443 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 294/584] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:50.205443 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10b196-e530-4fd9-9a5d-2a00c805d486 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:50.242192 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a9827-e27f-4642-8387-56ceef9bbf71 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:50.258299 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:50.261320 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:50.292297 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b3f5b-682b-4add-aaf5-a93e67a4b23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:50.293229 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b3f5b-682b-4add-aaf5-a93e67a4b23b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:50.293697 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b3f5b-682b-4add-aaf5-a93e67a4b23b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:50.294245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b3f5b-682b-4add-aaf5-a93e67a4b23b None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:50.295365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b3f5b-682b-4add-aaf5-a93e67a4b23b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:50.295827 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b3f5b-682b-4add-aaf5-a93e67a4b23b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:50.297556 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b3f5b-682b-4add-aaf5-a93e67a4b23b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:50.312797 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10b196-e530-4fd9-9a5d-2a00c805d486 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:50.317688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 291/585] 10.222.0.94 () {56 vars in 1137 bytes} [Mon Jul 18 13:55:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:50.318736 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b3f5b-682b-4add-aaf5-a93e67a4b23b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:50.351050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe725c1c-48b8-4b64-8714-9bfd0ebfba04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:50.352992 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe725c1c-48b8-4b64-8714-9bfd0ebfba04 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:50.353384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe725c1c-48b8-4b64-8714-9bfd0ebfba04 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:50.353783 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe725c1c-48b8-4b64-8714-9bfd0ebfba04 None admin] PATH_INFO: `/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/ac717fcd9fc1499db12cf852d45572a8/roles/5c4ba827ac854bc69cc26c3bcd27dc5c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:50.369373 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe725c1c-48b8-4b64-8714-9bfd0ebfba04 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec94946acf91486ba3f5b4fe834b8076, user_id=ac717fcd9fc1499db12cf852d45572a8, role_id=5c4ba827ac854bc69cc26c3bcd27dc5c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:50.370901 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe725c1c-48b8-4b64-8714-9bfd0ebfba04 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:50.385718 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 295/586] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:55:50 2022] PUT /identity/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/ac717fcd9fc1499db12cf852d45572a8/roles/5c4ba827ac854bc69cc26c3bcd27dc5c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:55:50.386358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe725c1c-48b8-4b64-8714-9bfd0ebfba04 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:50.453820 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b3f5b-682b-4add-aaf5-a93e67a4b23b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:50.506276 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:50.510148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c505a6-c50f-4a28-b630-4afc2b852357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:50.511566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c505a6-c50f-4a28-b630-4afc2b852357 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:50.512154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c505a6-c50f-4a28-b630-4afc2b852357 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:50.512719 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c505a6-c50f-4a28-b630-4afc2b852357 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:50.512967 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe725c1c-48b8-4b64-8714-9bfd0ebfba04 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:50.530572 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c505a6-c50f-4a28-b630-4afc2b852357 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:50.532699 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c505a6-c50f-4a28-b630-4afc2b852357 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:50.557015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-76c505a6-c50f-4a28-b630-4afc2b852357 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:50.557015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:50.557015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:50.557015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:50.557015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:50.557015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:50.557015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:50.557015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:50.557015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:50.557015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:50.557015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:50.557015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:50.557015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:50.557015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:50.557015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:50.557958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:50.557958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:50.557958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:50.557958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:50.557958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:50.557958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:50.557958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:50.557958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:50.557958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:50.557958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:50.557958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:50.557958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:50.557958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:50.557958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:50.557958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:50.557958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:50.557958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:50.559448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:50.559448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:50.559448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:50.559448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:50.559448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 292/587] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:50.559732 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c505a6-c50f-4a28-b630-4afc2b852357 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:50.566958 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96572b19-a290-4208-bb24-025d7ee969ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:50.567780 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96572b19-a290-4208-bb24-025d7ee969ee None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:50.568102 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96572b19-a290-4208-bb24-025d7ee969ee None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:50.568389 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96572b19-a290-4208-bb24-025d7ee969ee None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:50.578748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96572b19-a290-4208-bb24-025d7ee969ee None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:50.580810 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96572b19-a290-4208-bb24-025d7ee969ee None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:50.592386 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:50.601128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-96572b19-a290-4208-bb24-025d7ee969ee None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 13:55:50.601128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:50.601128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:50.601128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:50.601128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:50.601128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:50.601128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:50.601128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:50.601128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:50.601128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:50.601128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:50.601128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:50.601128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:55:50.601128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:55:50.601128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:55:50.601128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:55:50.602140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:50.602140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:50.602140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:55:50.602140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:50.602140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:55:50.602140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:50.602140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:50.602140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:50.602140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:50.602140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:50.602140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:50.602140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:50.602140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:50.602140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:50.602140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:50.602140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:50.602140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:50.602140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:50.603073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:50.603073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:50.603073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:55:50.603073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:55:50.603073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:55:50.603073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:55:50.603073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:55:50.603073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:55:50.603073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 13:55:50.603073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:50.603073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 296/588] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:50.603662 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96572b19-a290-4208-bb24-025d7ee969ee None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:50.678011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c505a6-c50f-4a28-b630-4afc2b852357 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:50.715678 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82032a8f-4f8e-4f7e-9a83-1525f814f547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:50.716915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82032a8f-4f8e-4f7e-9a83-1525f814f547 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:50.717426 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82032a8f-4f8e-4f7e-9a83-1525f814f547 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:50.717922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82032a8f-4f8e-4f7e-9a83-1525f814f547 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:50.729178 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82032a8f-4f8e-4f7e-9a83-1525f814f547 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:50.730967 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82032a8f-4f8e-4f7e-9a83-1525f814f547 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:50.731079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96572b19-a290-4208-bb24-025d7ee969ee None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:50.750669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-82032a8f-4f8e-4f7e-9a83-1525f814f547 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:50.750669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:50.750669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:50.750669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:50.750669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:50.750669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:50.750669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:50.750669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:50.750669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:50.750669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:50.750669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:50.750669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:50.750669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:50.750669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:50.750669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:50.751602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:50.751602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:50.751602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:50.751602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:50.751602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:50.751602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:50.751602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:50.751602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:50.751602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:50.751602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:50.751602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:50.751602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:50.751602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:50.751602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:50.751602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:50.751602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:50.751602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:50.752587 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:50.752587 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:50.752587 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:50.752587 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:50.759860 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 293/589] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:50.760998 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82032a8f-4f8e-4f7e-9a83-1525f814f547 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:50.770551 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169494b-33f3-4bf9-9035-43fd34ceb77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:50.771224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169494b-33f3-4bf9-9035-43fd34ceb77f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:50.771517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169494b-33f3-4bf9-9035-43fd34ceb77f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:50.771799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169494b-33f3-4bf9-9035-43fd34ceb77f None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:50.773017 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169494b-33f3-4bf9-9035-43fd34ceb77f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:50.773341 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169494b-33f3-4bf9-9035-43fd34ceb77f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:50.775750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169494b-33f3-4bf9-9035-43fd34ceb77f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:50.795754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 297/590] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:50.795754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169494b-33f3-4bf9-9035-43fd34ceb77f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:50.824342 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:50.851092 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:50.879622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82032a8f-4f8e-4f7e-9a83-1525f814f547 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:50.916854 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779e871-4452-4591-b71f-c2f5e2f3714b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:50.917765 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779e871-4452-4591-b71f-c2f5e2f3714b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:50.918251 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779e871-4452-4591-b71f-c2f5e2f3714b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:50.918681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779e871-4452-4591-b71f-c2f5e2f3714b None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:50.919668 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779e871-4452-4591-b71f-c2f5e2f3714b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:50.920169 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779e871-4452-4591-b71f-c2f5e2f3714b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:50.921737 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779e871-4452-4591-b71f-c2f5e2f3714b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:50.923702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169494b-33f3-4bf9-9035-43fd34ceb77f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:50.950076 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 294/591] 10.222.0.94 () {56 vars in 1137 bytes} [Mon Jul 18 13:55:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:50.951151 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779e871-4452-4591-b71f-c2f5e2f3714b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:50.970024 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d89f92-3e24-4010-978a-8d4f700f218b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:50.971057 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d89f92-3e24-4010-978a-8d4f700f218b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:50.971489 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d89f92-3e24-4010-978a-8d4f700f218b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:50.972380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d89f92-3e24-4010-978a-8d4f700f218b None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:50.973588 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d89f92-3e24-4010-978a-8d4f700f218b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:50.974048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d89f92-3e24-4010-978a-8d4f700f218b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:50.976042 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d89f92-3e24-4010-978a-8d4f700f218b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:50.996129 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 298/592] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:50.998009 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d89f92-3e24-4010-978a-8d4f700f218b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:51.082828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779e871-4452-4591-b71f-c2f5e2f3714b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:51.100221 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:51.124192 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:51.130282 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d89f92-3e24-4010-978a-8d4f700f218b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:51.140564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25931f-a4ab-44c1-89bf-ab2a2818172e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:51.142023 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25931f-a4ab-44c1-89bf-ab2a2818172e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:51.142904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25931f-a4ab-44c1-89bf-ab2a2818172e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:51.143354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25931f-a4ab-44c1-89bf-ab2a2818172e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:51.154588 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab25931f-a4ab-44c1-89bf-ab2a2818172e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:51.156959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab25931f-a4ab-44c1-89bf-ab2a2818172e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:51.167654 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be64c6-3a40-43f7-9d98-cc3d585bf21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:51.168612 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be64c6-3a40-43f7-9d98-cc3d585bf21d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:51.169043 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be64c6-3a40-43f7-9d98-cc3d585bf21d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:51.169414 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be64c6-3a40-43f7-9d98-cc3d585bf21d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:51.179035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-ab25931f-a4ab-44c1-89bf-ab2a2818172e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 13:55:51.179035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:51.179035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:51.179035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:51.179035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:51.179035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:51.179035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:51.179035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:51.179035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:51.179035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:51.179035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:51.179035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:51.179035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:55:51.179035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:55:51.179035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:55:51.179035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:55:51.180145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:51.180145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:51.180145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:55:51.180145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:51.180145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:55:51.180145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:51.180145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:51.180145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:51.180145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:51.180145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:51.180145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:51.180145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:51.180145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:51.180145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:51.180145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:51.180145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:51.180145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:51.180145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:51.181111 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:51.181111 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:51.181111 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:55:51.181111 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:55:51.181111 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:55:51.181111 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:55:51.181111 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:55:51.181111 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:55:51.181111 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 13:55:51.181111 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:51.181111 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 295/593] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:55:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:51.182101 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25931f-a4ab-44c1-89bf-ab2a2818172e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:51.185394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be64c6-3a40-43f7-9d98-cc3d585bf21d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:51.186940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be64c6-3a40-43f7-9d98-cc3d585bf21d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:51.205671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-31be64c6-3a40-43f7-9d98-cc3d585bf21d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:51.205671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:51.205671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:51.205671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:51.205671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:51.205671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:51.205671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:51.205671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:51.205671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:51.205671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:51.205671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:51.205671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:51.205671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:51.205671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:51.205671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:51.206618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:51.206618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:51.206618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:51.206618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:51.206618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:51.206618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:51.206618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:51.206618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:51.206618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:51.206618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:51.206618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:51.206618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:51.206618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:51.206618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:51.206618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:51.206618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:51.206618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:51.207506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:51.207506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:51.207506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:51.207506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:51.207506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 299/594] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:51.208232 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be64c6-3a40-43f7-9d98-cc3d585bf21d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:51.295260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25931f-a4ab-44c1-89bf-ab2a2818172e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:51.312948 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be64c6-3a40-43f7-9d98-cc3d585bf21d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:51.331223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c021e57-f6e2-450a-871a-e65fa7fe2ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:51.332050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c021e57-f6e2-450a-871a-e65fa7fe2ecb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:51.332344 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c021e57-f6e2-450a-871a-e65fa7fe2ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:51.332632 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c021e57-f6e2-450a-871a-e65fa7fe2ecb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:51.339710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c021e57-f6e2-450a-871a-e65fa7fe2ecb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:51.341215 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c021e57-f6e2-450a-871a-e65fa7fe2ecb None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:51.342733 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:51.350148 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97093a2-63c8-4808-846c-56487ae2ca9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:51.350987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97093a2-63c8-4808-846c-56487ae2ca9a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:51.351433 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97093a2-63c8-4808-846c-56487ae2ca9a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:51.352434 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97093a2-63c8-4808-846c-56487ae2ca9a None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:51.353415 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97093a2-63c8-4808-846c-56487ae2ca9a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:51.353870 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97093a2-63c8-4808-846c-56487ae2ca9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:51.355422 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97093a2-63c8-4808-846c-56487ae2ca9a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:51.356897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-5c021e57-f6e2-450a-871a-e65fa7fe2ecb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:51.356897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:51.356897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:51.356897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:51.356897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:51.356897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:51.356897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:51.356897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:51.356897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:51.356897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:51.356897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:51.356897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:51.356897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:55:51.356897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:55:51.356897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:55:51.357910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:55:51.357910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:51.357910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:51.357910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:51.357910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:51.357910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:51.357910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:51.357910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:51.357910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:51.357910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:51.357910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:51.357910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:51.357910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:51.357910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:51.357910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:51.357910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:51.357910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:51.359002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:55:51.359002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:55:51.359002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:55:51.359002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:55:51.359002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:55:51.359002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:55:51.359002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:51.359002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:51.362778 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 296/595] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:51.363615 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c021e57-f6e2-450a-871a-e65fa7fe2ecb None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:51.377320 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 300/596] 10.222.0.94 () {56 vars in 1139 bytes} [Mon Jul 18 13:55:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:51.382629 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97093a2-63c8-4808-846c-56487ae2ca9a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:51.420143 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:51.470566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c021e57-f6e2-450a-871a-e65fa7fe2ecb None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:51.487995 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97093a2-63c8-4808-846c-56487ae2ca9a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:51.506512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2d875-a43c-4709-886b-18a1d6a0353b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:51.507206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2d875-a43c-4709-886b-18a1d6a0353b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:51.507498 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2d875-a43c-4709-886b-18a1d6a0353b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:51.507791 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2d875-a43c-4709-886b-18a1d6a0353b None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:51.508610 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2d875-a43c-4709-886b-18a1d6a0353b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:51.508928 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2d875-a43c-4709-886b-18a1d6a0353b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:51.510532 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2d875-a43c-4709-886b-18a1d6a0353b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:51.524452 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 297/597] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:51.525566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2d875-a43c-4709-886b-18a1d6a0353b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:51.528269 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6139fc-75b6-4cfb-8d9f-d83f01df9a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:51.529152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6139fc-75b6-4cfb-8d9f-d83f01df9a0d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:51.529577 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6139fc-75b6-4cfb-8d9f-d83f01df9a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:51.529995 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6139fc-75b6-4cfb-8d9f-d83f01df9a0d None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:51.530929 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6139fc-75b6-4cfb-8d9f-d83f01df9a0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:51.531427 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6139fc-75b6-4cfb-8d9f-d83f01df9a0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:51.533656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6139fc-75b6-4cfb-8d9f-d83f01df9a0d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:51.547501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 301/598] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:55:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:51.548528 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6139fc-75b6-4cfb-8d9f-d83f01df9a0d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:51.604093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:51.636553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2d875-a43c-4709-886b-18a1d6a0353b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:51.651027 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:51.658225 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6139fc-75b6-4cfb-8d9f-d83f01df9a0d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:51.673642 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf5c8e-65a5-4733-a50d-6ab5db4dbca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:51.674408 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf5c8e-65a5-4733-a50d-6ab5db4dbca5 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:51.674700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf5c8e-65a5-4733-a50d-6ab5db4dbca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:51.674983 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf5c8e-65a5-4733-a50d-6ab5db4dbca5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:51.686089 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf5c8e-65a5-4733-a50d-6ab5db4dbca5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:51.687718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf5c8e-65a5-4733-a50d-6ab5db4dbca5 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:51.694613 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd7fb0-29e8-4423-96fc-4825670f9a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:51.695739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd7fb0-29e8-4423-96fc-4825670f9a55 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:51.696192 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd7fb0-29e8-4423-96fc-4825670f9a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:51.696628 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd7fb0-29e8-4423-96fc-4825670f9a55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:51.703700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd7fb0-29e8-4423-96fc-4825670f9a55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:51.705665 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd7fb0-29e8-4423-96fc-4825670f9a55 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:51.706959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-33cf5c8e-65a5-4733-a50d-6ab5db4dbca5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:51.706959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:51.706959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:51.706959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:51.706959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:51.706959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:51.706959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:51.706959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:51.706959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:51.706959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:51.706959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:51.706959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:51.706959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:51.706959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:51.706959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:51.708004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:51.708004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:51.708004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:51.708004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:51.708004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:51.708004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:51.708004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:51.708004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:51.708004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:51.708004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:51.708004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:51.708004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:51.708004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:51.708004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:51.708004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:51.708004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:51.708004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:51.709109 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:51.709109 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:51.709109 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:51.709109 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:51.713390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 298/599] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:51.714391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf5c8e-65a5-4733-a50d-6ab5db4dbca5 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:51.723866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-73cd7fb0-29e8-4423-96fc-4825670f9a55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:51.723866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:51.723866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:51.723866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:51.723866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:51.723866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:51.723866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:51.723866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:51.723866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:51.723866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:51.723866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:51.723866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:51.723866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:55:51.723866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:55:51.723866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:55:51.724914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:55:51.724914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:51.724914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:51.724914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:51.724914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:51.724914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:51.724914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:51.724914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:51.724914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:51.724914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:51.724914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:51.724914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:51.724914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:51.724914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:51.724914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:51.724914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:51.724914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:51.725992 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:55:51.725992 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:55:51.725992 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:55:51.725992 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:55:51.725992 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:55:51.725992 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:55:51.725992 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:51.725992 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:51.728157 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 302/600] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:51.729178 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd7fb0-29e8-4423-96fc-4825670f9a55 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:51.820895 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf5c8e-65a5-4733-a50d-6ab5db4dbca5 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:51.830970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd7fb0-29e8-4423-96fc-4825670f9a55 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:51.852347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:51.856963 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874db7b-9fae-4a4e-9bd0-8cfc8d436bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:51.857864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874db7b-9fae-4a4e-9bd0-8cfc8d436bf8 None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:51.858153 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874db7b-9fae-4a4e-9bd0-8cfc8d436bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:51.858434 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874db7b-9fae-4a4e-9bd0-8cfc8d436bf8 None admin] PATH_INFO: `/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/503a8228d4754c309fc3f365f4cfc61b/roles/5c4ba827ac854bc69cc26c3bcd27dc5c` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:51.868511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714fab4d-7167-4d98-82dc-28fd9b2c8915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:51.869387 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fab4d-7167-4d98-82dc-28fd9b2c8915 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:51.870024 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fab4d-7167-4d98-82dc-28fd9b2c8915 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:51.870448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fab4d-7167-4d98-82dc-28fd9b2c8915 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:51.871318 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714fab4d-7167-4d98-82dc-28fd9b2c8915 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:51.871823 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714fab4d-7167-4d98-82dc-28fd9b2c8915 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:51.873553 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714fab4d-7167-4d98-82dc-28fd9b2c8915 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:51.875639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4874db7b-9fae-4a4e-9bd0-8cfc8d436bf8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec94946acf91486ba3f5b4fe834b8076, user_id=503a8228d4754c309fc3f365f4cfc61b, role_id=5c4ba827ac854bc69cc26c3bcd27dc5c)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:51.877125 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4874db7b-9fae-4a4e-9bd0-8cfc8d436bf8 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:51.887439 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 303/601] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:51.888612 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714fab4d-7167-4d98-82dc-28fd9b2c8915 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:51.900119 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 299/602] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:55:51 2022] PUT /identity/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/503a8228d4754c309fc3f365f4cfc61b/roles/5c4ba827ac854bc69cc26c3bcd27dc5c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:55:51.901284 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874db7b-9fae-4a4e-9bd0-8cfc8d436bf8 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:51.940570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:52.091834 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714fab4d-7167-4d98-82dc-28fd9b2c8915 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:52.123834 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874db7b-9fae-4a4e-9bd0-8cfc8d436bf8 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:52.134187 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce156a1-4d0f-4c76-9603-e3c8866f9682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:52.135092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce156a1-4d0f-4c76-9603-e3c8866f9682 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:52.135542 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce156a1-4d0f-4c76-9603-e3c8866f9682 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:52.136002 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce156a1-4d0f-4c76-9603-e3c8866f9682 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:52.137123 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce156a1-4d0f-4c76-9603-e3c8866f9682 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:52.137617 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce156a1-4d0f-4c76-9603-e3c8866f9682 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:52.139275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce156a1-4d0f-4c76-9603-e3c8866f9682 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:52.153057 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 304/603] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:55:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:52.161022 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce156a1-4d0f-4c76-9603-e3c8866f9682 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:52.165773 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d1642-14b0-4d60-8e23-60b921d2d441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:52.166589 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d1642-14b0-4d60-8e23-60b921d2d441 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:52.166876 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d1642-14b0-4d60-8e23-60b921d2d441 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:52.167157 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d1642-14b0-4d60-8e23-60b921d2d441 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:52.174459 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d1642-14b0-4d60-8e23-60b921d2d441 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:52.176057 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d1642-14b0-4d60-8e23-60b921d2d441 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:52.192845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-ab4d1642-14b0-4d60-8e23-60b921d2d441 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:52.192845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:52.192845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:52.192845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:52.192845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:52.192845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:52.192845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:52.192845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:52.192845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:52.192845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:52.192845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:52.192845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:52.192845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:55:52.192845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:55:52.192845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:55:52.193922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:55:52.193922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:52.193922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:52.193922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:52.193922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:52.193922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:52.193922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:52.193922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:52.193922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:52.193922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:52.193922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:52.193922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:52.193922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:52.193922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:52.193922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:52.193922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:52.193922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:52.194989 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:55:52.194989 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:55:52.194989 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:55:52.194989 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:55:52.194989 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:55:52.194989 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:55:52.194989 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:52.194989 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:52.194989 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 300/604] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:52.200288 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:52.203998 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d1642-14b0-4d60-8e23-60b921d2d441 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:52.287607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:52.291979 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce156a1-4d0f-4c76-9603-e3c8866f9682 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:52.315379 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d1642-14b0-4d60-8e23-60b921d2d441 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:52.338321 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b7f62a-3893-4df3-896f-e6aedcdd02d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:52.339700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7f62a-3893-4df3-896f-e6aedcdd02d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:52.340161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7f62a-3893-4df3-896f-e6aedcdd02d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:52.340834 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7f62a-3893-4df3-896f-e6aedcdd02d5 None admin] PATH_INFO: `/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/45ee15f7716541a1a61443069d5cf4f8/roles/5c4ba827ac854bc69cc26c3bcd27dc5c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:52.352676 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868ec39-9f94-4e5b-86c9-466eb036b9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:52.353416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868ec39-9f94-4e5b-86c9-466eb036b9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:52.353874 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868ec39-9f94-4e5b-86c9-466eb036b9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:52.354266 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868ec39-9f94-4e5b-86c9-466eb036b9c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:52.355169 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3868ec39-9f94-4e5b-86c9-466eb036b9c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:52.355597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3868ec39-9f94-4e5b-86c9-466eb036b9c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:52.357167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3868ec39-9f94-4e5b-86c9-466eb036b9c3 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:52.361259 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b7f62a-3893-4df3-896f-e6aedcdd02d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec94946acf91486ba3f5b4fe834b8076, user_id=45ee15f7716541a1a61443069d5cf4f8, role_id=5c4ba827ac854bc69cc26c3bcd27dc5c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:52.362854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b7f62a-3893-4df3-896f-e6aedcdd02d5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:52.371993 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 301/605] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:55:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:52.373490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae430cbd-26f0-4bdf-a065-e350b0261f22 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:52.373974 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae430cbd-26f0-4bdf-a065-e350b0261f22 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:52.374400 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae430cbd-26f0-4bdf-a065-e350b0261f22 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:52.379914 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 302/606] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:52.380838 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae430cbd-26f0-4bdf-a065-e350b0261f22 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:52.381933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 305/607] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:55:52 2022] PUT /identity/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/45ee15f7716541a1a61443069d5cf4f8/roles/5c4ba827ac854bc69cc26c3bcd27dc5c => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:55:52.384107 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e75457-82a3-4263-8db4-4196cba2e260 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:52.384853 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e75457-82a3-4263-8db4-4196cba2e260 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:52.385302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e75457-82a3-4263-8db4-4196cba2e260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:52.408865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-31e75457-82a3-4263-8db4-4196cba2e260 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:52.410696 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-31e75457-82a3-4263-8db4-4196cba2e260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:52.496988 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 306/608] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:52.500787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae430cbd-26f0-4bdf-a065-e350b0261f22 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:52.522194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e75457-82a3-4263-8db4-4196cba2e260 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:52.534626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:52.564163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c792a55-ad63-423f-98ba-a9dfcc9076ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:52.565079 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c792a55-ad63-423f-98ba-a9dfcc9076ab None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:52.565371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c792a55-ad63-423f-98ba-a9dfcc9076ab None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:52.565683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c792a55-ad63-423f-98ba-a9dfcc9076ab None admin] PATH_INFO: `/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/14027a0e13fd4272bd7803e3131b6348/roles/5c4ba827ac854bc69cc26c3bcd27dc5c` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:52.569037 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:52.584865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c792a55-ad63-423f-98ba-a9dfcc9076ab None admin] RBAC: Authorizing `identity:create_grant(project_id=ec94946acf91486ba3f5b4fe834b8076, user_id=14027a0e13fd4272bd7803e3131b6348, role_id=5c4ba827ac854bc69cc26c3bcd27dc5c)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:52.587017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c792a55-ad63-423f-98ba-a9dfcc9076ab None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:52.604360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 303/609] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:55:52 2022] PUT /identity/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/14027a0e13fd4272bd7803e3131b6348/roles/5c4ba827ac854bc69cc26c3bcd27dc5c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:55:52.604942 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab62932-ba1d-4cde-a551-e551c2c2112d None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:52.605240 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab62932-ba1d-4cde-a551-e551c2c2112d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:52.605542 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab62932-ba1d-4cde-a551-e551c2c2112d None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:52.606856 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 304/610] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:52.613645 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8adb68-bc6b-4fd5-b85f-7878117284fe None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:52.614079 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8adb68-bc6b-4fd5-b85f-7878117284fe None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:52.614495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8adb68-bc6b-4fd5-b85f-7878117284fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:52.634778 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-9c8adb68-bc6b-4fd5-b85f-7878117284fe None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:52.636197 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9c8adb68-bc6b-4fd5-b85f-7878117284fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:52.680013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e75457-82a3-4263-8db4-4196cba2e260 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:52.725272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b1ffea-1935-4d54-94a0-a58de7fc1648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:52.726419 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1ffea-1935-4d54-94a0-a58de7fc1648 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:52.727451 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1ffea-1935-4d54-94a0-a58de7fc1648 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:52.727889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1ffea-1935-4d54-94a0-a58de7fc1648 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:52.732222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 305/611] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:52.751656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b1ffea-1935-4d54-94a0-a58de7fc1648 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:52.757053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8adb68-bc6b-4fd5-b85f-7878117284fe None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:52.761481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-94b1ffea-1935-4d54-94a0-a58de7fc1648 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:52.761481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:52.761481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:52.761481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:52.761481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:52.761481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:52.761481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:52.761481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:52.761481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:52.761481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:52.761481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:52.761481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:52.761481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:52.761481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:52.761481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:52.761481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:52.762437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:52.762437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:52.762437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:52.762437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:52.762437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:52.762437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:52.762437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:52.762437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:52.762437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:52.762437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:52.762437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:52.762437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:52.762437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:52.762437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:52.762437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:52.762437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:52.764401 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:52.764401 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:52.764401 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:52.764401 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:52.764401 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:52.764401 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:52.764401 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:52.764401 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:52.764401 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:52.764401 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:52.764401 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:52.764401 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:52.764401 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 307/612] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:55:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:55:52.769007 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b1ffea-1935-4d54-94a0-a58de7fc1648 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:52.874136 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:52.932215 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b1ffea-1935-4d54-94a0-a58de7fc1648 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:52.951973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8adb68-bc6b-4fd5-b85f-7878117284fe None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:52.991882 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:53.010038 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae9e3e-9893-4842-a4b5-ddab778e9e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:53.010717 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae9e3e-9893-4842-a4b5-ddab778e9e38 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:53.011008 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae9e3e-9893-4842-a4b5-ddab778e9e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:53.011289 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae9e3e-9893-4842-a4b5-ddab778e9e38 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:53.012091 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae9e3e-9893-4842-a4b5-ddab778e9e38 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:53.012405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae9e3e-9893-4842-a4b5-ddab778e9e38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:53.013789 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae9e3e-9893-4842-a4b5-ddab778e9e38 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:53.028318 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 308/613] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Jul 18 13:55:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:53.028969 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8792aa-f25a-4951-a120-6c80d9ee46c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:53.029783 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8792aa-f25a-4951-a120-6c80d9ee46c9 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:53.030078 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8792aa-f25a-4951-a120-6c80d9ee46c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:53.030367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8792aa-f25a-4951-a120-6c80d9ee46c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:53.036335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae9e3e-9893-4842-a4b5-ddab778e9e38 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:53.055995 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8792aa-f25a-4951-a120-6c80d9ee46c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:53.060488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-fd8792aa-f25a-4951-a120-6c80d9ee46c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:53.060488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:53.060488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:53.060488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:53.060488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:53.060488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:53.060488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:53.060488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:53.060488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:53.060488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:53.060488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:53.060488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:53.060488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:53.060488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:53.060488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:53.060488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:53.061418 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:53.061418 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:53.061418 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:53.061418 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:53.061418 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:53.061418 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:53.061418 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:53.061418 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:53.061418 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:53.061418 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:53.061418 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:53.061418 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:53.061418 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:53.061418 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:53.061418 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:53.061418 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:53.062343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:53.062343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:53.062343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:53.062343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:53.062343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:53.062343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:53.062343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:53.062343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:53.062343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:53.062343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:53.062343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:53.062343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:53.062343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 306/614] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:55:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:53.067731 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8792aa-f25a-4951-a120-6c80d9ee46c9 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:53.198890 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae9e3e-9893-4842-a4b5-ddab778e9e38 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:53.254395 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:53.264522 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8792aa-f25a-4951-a120-6c80d9ee46c9 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:53.267569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e049bf4-bed5-4c89-8014-3704d938e6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:53.268404 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e049bf4-bed5-4c89-8014-3704d938e6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:53.268695 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e049bf4-bed5-4c89-8014-3704d938e6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:53.268978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e049bf4-bed5-4c89-8014-3704d938e6c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:53.302744 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e049bf4-bed5-4c89-8014-3704d938e6c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:53.304265 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e049bf4-bed5-4c89-8014-3704d938e6c4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:53.309156 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c183f31-2da6-4b6a-a551-dde599b38ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:53.309836 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c183f31-2da6-4b6a-a551-dde599b38ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:53.310125 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c183f31-2da6-4b6a-a551-dde599b38ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:53.310406 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c183f31-2da6-4b6a-a551-dde599b38ef5 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:53.311163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c183f31-2da6-4b6a-a551-dde599b38ef5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:53.311474 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c183f31-2da6-4b6a-a551-dde599b38ef5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:53.315056 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c183f31-2da6-4b6a-a551-dde599b38ef5 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:53.333080 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 307/615] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Jul 18 13:55:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:53.336890 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c183f31-2da6-4b6a-a551-dde599b38ef5 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:53.341521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6e049bf4-bed5-4c89-8014-3704d938e6c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:53.341521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:53.341521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:53.341521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:53.341521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:53.341521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:53.341521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:53.341521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:53.341521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:53.341521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:53.341521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:53.341521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:53.341521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:53.341521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:53.341521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:53.342418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:53.342418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:53.342418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:53.342418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:53.342418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:53.342418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:53.342418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:53.342418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:53.342418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:53.342418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:53.342418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:53.342418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:53.342418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:53.342418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:53.342418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:53.342418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:53.342418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:53.343314 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:53.343314 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:53.343314 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:53.343314 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:53.343314 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 309/616] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:53.349571 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e049bf4-bed5-4c89-8014-3704d938e6c4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:53.407889 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:53.525336 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c183f31-2da6-4b6a-a551-dde599b38ef5 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:53.548394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e049bf4-bed5-4c89-8014-3704d938e6c4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:53.595365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e289d76-04ca-4dfd-bce3-dd983e4616d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:53.596194 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e289d76-04ca-4dfd-bce3-dd983e4616d4 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:53.596484 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e289d76-04ca-4dfd-bce3-dd983e4616d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:53.596761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e289d76-04ca-4dfd-bce3-dd983e4616d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:53.614650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e289d76-04ca-4dfd-bce3-dd983e4616d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:53.617296 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e289d76-04ca-4dfd-bce3-dd983e4616d4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:53.618433 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98434045-29b0-47a6-b732-2e081039cb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:53.619250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98434045-29b0-47a6-b732-2e081039cb6f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:53.619553 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98434045-29b0-47a6-b732-2e081039cb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:53.619832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98434045-29b0-47a6-b732-2e081039cb6f None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:53.620670 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98434045-29b0-47a6-b732-2e081039cb6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:53.620984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98434045-29b0-47a6-b732-2e081039cb6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:53.622646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98434045-29b0-47a6-b732-2e081039cb6f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:53.652745 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 310/617] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:53.652925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-6e289d76-04ca-4dfd-bce3-dd983e4616d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:53.652925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:53.652925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:53.652925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:53.652925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:53.652925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:53.652925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:53.652925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:53.652925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:53.652925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:53.652925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:53.652925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:53.652925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:53.652925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:53.652925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:53.653785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:53.653785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:53.653785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:53.653785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:53.653785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:53.653785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:53.653785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:53.653785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:53.653785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:53.653785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:53.653785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:53.653785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:53.653785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:53.653785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:53.653785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:53.653785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:53.653785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:53.654677 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:53.654677 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:53.654677 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:53.654677 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:53.654677 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 308/618] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:53.660949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98434045-29b0-47a6-b732-2e081039cb6f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:53.665998 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e289d76-04ca-4dfd-bce3-dd983e4616d4 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:53.736483 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:53.806454 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98434045-29b0-47a6-b732-2e081039cb6f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:53.843252 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:53.864542 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e289d76-04ca-4dfd-bce3-dd983e4616d4 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:53.871009 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52140825-ef3d-4650-8c7d-2287b33173d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:53.871821 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52140825-ef3d-4650-8c7d-2287b33173d5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:53.872134 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52140825-ef3d-4650-8c7d-2287b33173d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:53.872408 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52140825-ef3d-4650-8c7d-2287b33173d5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:53.885871 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52140825-ef3d-4650-8c7d-2287b33173d5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:53.887331 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52140825-ef3d-4650-8c7d-2287b33173d5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:53.907380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-52140825-ef3d-4650-8c7d-2287b33173d5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 13:55:53.907380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:53.907380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:53.907380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:53.907380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:53.907380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:53.907380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:53.907380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:53.907380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:53.907380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:53.907380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:53.907380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:53.907380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:55:53.907380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:55:53.907380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:55:53.907380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:55:53.908463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:53.908463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:53.908463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:55:53.908463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:53.908463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:55:53.908463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:53.908463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:53.908463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:53.908463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:53.908463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:53.908463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:53.908463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:53.908463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:53.908463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:53.908463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:53.908463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:53.908463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:53.908463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:53.909569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:53.909569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:53.909569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:55:53.909569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:55:53.909569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:55:53.909569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:55:53.909569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:55:53.909569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:55:53.909569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 13:55:53.909569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:53.915281 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 311/619] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:55:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:53.919227 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52140825-ef3d-4650-8c7d-2287b33173d5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:53.933364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a4216-23bb-4abf-b6f7-dd3aa2aa5ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:53.934034 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a4216-23bb-4abf-b6f7-dd3aa2aa5ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:53.934320 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a4216-23bb-4abf-b6f7-dd3aa2aa5ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:53.934598 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a4216-23bb-4abf-b6f7-dd3aa2aa5ba4 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:53.935320 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a4216-23bb-4abf-b6f7-dd3aa2aa5ba4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:53.935627 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a4216-23bb-4abf-b6f7-dd3aa2aa5ba4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:53.937037 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a4216-23bb-4abf-b6f7-dd3aa2aa5ba4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:53.955474 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 309/620] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:53.961724 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a4216-23bb-4abf-b6f7-dd3aa2aa5ba4 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:54.104917 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52140825-ef3d-4650-8c7d-2287b33173d5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:54.107340 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:54.139440 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a4216-23bb-4abf-b6f7-dd3aa2aa5ba4 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:54.172686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef32a51-534e-4aeb-ad71-2c51e4f3467e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:54.173379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef32a51-534e-4aeb-ad71-2c51e4f3467e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:54.173669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef32a51-534e-4aeb-ad71-2c51e4f3467e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:54.173946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef32a51-534e-4aeb-ad71-2c51e4f3467e None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:54.174796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef32a51-534e-4aeb-ad71-2c51e4f3467e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:54.175109 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef32a51-534e-4aeb-ad71-2c51e4f3467e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:54.176607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef32a51-534e-4aeb-ad71-2c51e4f3467e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:54.184969 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464f324-88e3-4702-937e-ff66b4e42f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:54.185881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464f324-88e3-4702-937e-ff66b4e42f87 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:54.186390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464f324-88e3-4702-937e-ff66b4e42f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:54.186764 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464f324-88e3-4702-937e-ff66b4e42f87 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:54.194691 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0464f324-88e3-4702-937e-ff66b4e42f87 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:54.196405 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0464f324-88e3-4702-937e-ff66b4e42f87 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:54.214276 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 312/621] 10.222.0.94 () {56 vars in 1135 bytes} [Mon Jul 18 13:55:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:54.216594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-0464f324-88e3-4702-937e-ff66b4e42f87 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 13:55:54.216594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:54.216594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:54.216594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:54.216594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:54.216594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:54.216594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:54.216594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:54.216594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:54.216594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:54.216594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:54.216594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:54.216594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:55:54.216594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:55:54.216594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:55:54.216594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:55:54.217541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:54.217541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:54.217541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:55:54.217541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:54.217541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:55:54.217541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:54.217541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:54.217541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:54.217541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:54.217541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:54.217541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:54.217541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:54.217541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:54.217541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:54.217541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:54.217541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:54.217541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:54.217541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:54.218496 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:54.218496 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:54.218496 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:55:54.218496 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:55:54.218496 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:55:54.218496 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:55:54.218496 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:55:54.218496 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:55:54.218496 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 13:55:54.218496 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:54.218496 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 310/622] 10.222.0.94 () {56 vars in 1093 bytes} [Mon Jul 18 13:55:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:54.224694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef32a51-534e-4aeb-ad71-2c51e4f3467e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:54.226262 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464f324-88e3-4702-937e-ff66b4e42f87 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:54.251086 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:54.358020 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464f324-88e3-4702-937e-ff66b4e42f87 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:54.360462 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef32a51-534e-4aeb-ad71-2c51e4f3467e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:54.400131 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6dec87-1d2b-4c8a-9a3d-ff9671780769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:54.400790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6dec87-1d2b-4c8a-9a3d-ff9671780769 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:54.401076 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6dec87-1d2b-4c8a-9a3d-ff9671780769 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:54.401352 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6dec87-1d2b-4c8a-9a3d-ff9671780769 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:54.402297 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6dec87-1d2b-4c8a-9a3d-ff9671780769 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:54.402613 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6dec87-1d2b-4c8a-9a3d-ff9671780769 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:54.404064 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6dec87-1d2b-4c8a-9a3d-ff9671780769 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:54.407558 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70f444-f52c-40a0-aac6-c3af5e4a8904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:54.408662 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70f444-f52c-40a0-aac6-c3af5e4a8904 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:54.409157 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70f444-f52c-40a0-aac6-c3af5e4a8904 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:54.409662 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70f444-f52c-40a0-aac6-c3af5e4a8904 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:54.420771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70f444-f52c-40a0-aac6-c3af5e4a8904 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:54.426430 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 311/623] 10.222.0.94 () {56 vars in 1133 bytes} [Mon Jul 18 13:55:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:54.426430 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce852e-acc1-4dd6-b228-79ad4de4d6ca None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:54.426430 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce852e-acc1-4dd6-b228-79ad4de4d6ca None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:54.426430 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce852e-acc1-4dd6-b228-79ad4de4d6ca None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:54.426430 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 312/624] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:54.427294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce852e-acc1-4dd6-b228-79ad4de4d6ca None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:54.427769 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70f444-f52c-40a0-aac6-c3af5e4a8904 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:54.448423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ac70f444-f52c-40a0-aac6-c3af5e4a8904 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:54.448423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:54.448423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:54.448423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:54.448423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:54.448423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:54.448423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:54.448423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:54.448423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:54.448423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:54.448423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:54.448423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:54.448423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:54.448423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:54.448423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:54.449394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:54.449394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:54.449394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:54.449394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:54.449394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:54.449394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:54.449394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:54.449394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:54.449394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:54.449394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:54.449394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:54.449394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:54.449394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:54.449394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:54.449394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:54.449394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:54.449394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:54.450293 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:54.450293 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:54.450293 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:54.450293 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:54.452728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 313/625] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:54.454664 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cc228-488b-4978-bf69-5cbc571d6851 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:54.455183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cc228-488b-4978-bf69-5cbc571d6851 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:54.455783 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cc228-488b-4978-bf69-5cbc571d6851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:54.474742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2f8cc228-488b-4978-bf69-5cbc571d6851 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:54.476495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2f8cc228-488b-4978-bf69-5cbc571d6851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:54.485981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:54.498639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:54.536353 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce852e-acc1-4dd6-b228-79ad4de4d6ca None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:54.556310 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 314/626] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:54.557592 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8cc228-488b-4978-bf69-5cbc571d6851 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:54.572195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7021f3a5-341a-4d50-9a41-39a8cab1890b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:54.573084 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021f3a5-341a-4d50-9a41-39a8cab1890b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:54.573520 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021f3a5-341a-4d50-9a41-39a8cab1890b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:54.573940 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021f3a5-341a-4d50-9a41-39a8cab1890b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:54.643158 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7021f3a5-341a-4d50-9a41-39a8cab1890b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:54.645025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7021f3a5-341a-4d50-9a41-39a8cab1890b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:54.667558 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8cc228-488b-4978-bf69-5cbc571d6851 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:54.706414 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec743a34-9149-4797-9920-0bf4bf7e628b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:54.707294 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec743a34-9149-4797-9920-0bf4bf7e628b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:54.707505 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec743a34-9149-4797-9920-0bf4bf7e628b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:54.707754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec743a34-9149-4797-9920-0bf4bf7e628b None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:54.708546 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec743a34-9149-4797-9920-0bf4bf7e628b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:54.708826 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec743a34-9149-4797-9920-0bf4bf7e628b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:54.710228 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec743a34-9149-4797-9920-0bf4bf7e628b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:54.712437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:54.725140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 315/627] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:54.726043 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec743a34-9149-4797-9920-0bf4bf7e628b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:54.744854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:54.750669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 313/628] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:54.751832 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7021f3a5-341a-4d50-9a41-39a8cab1890b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:54.848665 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec743a34-9149-4797-9920-0bf4bf7e628b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:54.859769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7021f3a5-341a-4d50-9a41-39a8cab1890b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:54.883317 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a3f8c-d86e-4a6f-bab5-1eaa1cb6944c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:54.884141 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a3f8c-d86e-4a6f-bab5-1eaa1cb6944c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:54.884426 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a3f8c-d86e-4a6f-bab5-1eaa1cb6944c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:54.884700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a3f8c-d86e-4a6f-bab5-1eaa1cb6944c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:54.895208 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a3f8c-d86e-4a6f-bab5-1eaa1cb6944c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:54.896413 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf3ebc7-29f8-4fb7-956f-22257373f2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:54.897373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3ebc7-29f8-4fb7-956f-22257373f2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:54.897801 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3ebc7-29f8-4fb7-956f-22257373f2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:54.898220 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3ebc7-29f8-4fb7-956f-22257373f2a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:54.903287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3e4a3f8c-d86e-4a6f-bab5-1eaa1cb6944c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:54.903287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:54.903287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:54.903287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:54.903287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:54.903287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:54.903287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:54.903287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:54.903287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:54.903287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:54.903287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:54.903287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:54.903287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:54.903287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:54.903287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:54.903287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:54.904521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:54.904521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:54.904521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:54.904521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:54.904521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:54.904521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:54.904521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:54.904521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:54.904521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:54.904521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:54.904521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:54.904521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:54.904521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:54.904521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:54.904521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:54.904521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:54.905372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:54.905372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:54.905372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:54.905372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:54.905372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:54.905372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:54.905372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:54.905372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:54.905372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:54.905372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:54.905372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:54.905372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:54.905372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 316/629] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:55:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:54.906064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a3f8c-d86e-4a6f-bab5-1eaa1cb6944c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:54.908850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf3ebc7-29f8-4fb7-956f-22257373f2a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:54.910557 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf3ebc7-29f8-4fb7-956f-22257373f2a1 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:54.925594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-adf3ebc7-29f8-4fb7-956f-22257373f2a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:54.925594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:54.925594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:54.925594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:54.925594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:54.925594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:54.925594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:54.925594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:54.925594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:54.925594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:54.925594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:54.925594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:54.925594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:55:54.925594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:55:54.925594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:55:54.926536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:55:54.926536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:54.926536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:54.926536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:54.926536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:54.926536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:54.926536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:54.926536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:54.926536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:54.926536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:54.926536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:54.926536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:54.926536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:54.926536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:54.926536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:54.926536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:54.926536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:54.927472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:55:54.927472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:55:54.927472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:55:54.927472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:55:54.927472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:55:54.927472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:55:54.927472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:54.927472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:54.927472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 314/630] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:54.929457 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8c0e7-de74-4c09-9a82-e89861bc9db0 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:54.929457 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8c0e7-de74-4c09-9a82-e89861bc9db0 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:54.929457 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8c0e7-de74-4c09-9a82-e89861bc9db0 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:54.933548 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 315/631] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:54.933548 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8c0e7-de74-4c09-9a82-e89861bc9db0 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:54.959662 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:54.982235 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:55.020486 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a3f8c-d86e-4a6f-bab5-1eaa1cb6944c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:55.034813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8c0e7-de74-4c09-9a82-e89861bc9db0 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:55.054383 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab098f-a2ab-4f0e-b236-c7c18aed9883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:55.055433 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab098f-a2ab-4f0e-b236-c7c18aed9883 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:55.055716 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab098f-a2ab-4f0e-b236-c7c18aed9883 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:55.056014 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab098f-a2ab-4f0e-b236-c7c18aed9883 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:55.066756 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab098f-a2ab-4f0e-b236-c7c18aed9883 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:55.067796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d7560-3bce-4716-9d12-86cf334d06fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:55.068248 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab098f-a2ab-4f0e-b236-c7c18aed9883 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:55.068491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d7560-3bce-4716-9d12-86cf334d06fa None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:55.068780 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d7560-3bce-4716-9d12-86cf334d06fa None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:55.069061 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d7560-3bce-4716-9d12-86cf334d06fa None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:55.069805 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d7560-3bce-4716-9d12-86cf334d06fa None admin] RBAC: Adding query filter params (name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:55.070111 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d7560-3bce-4716-9d12-86cf334d06fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:55.071997 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d7560-3bce-4716-9d12-86cf334d06fa None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:55.080063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 316/632] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Jul 18 13:55:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:55.080839 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d7560-3bce-4716-9d12-86cf334d06fa None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:55.087799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-53ab098f-a2ab-4f0e-b236-c7c18aed9883 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:55.087799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:55.087799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:55.087799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:55.087799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:55.087799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:55.087799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:55.087799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:55.087799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:55.087799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:55.087799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:55.087799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:55.087799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:55.087799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:55.087799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:55.088892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:55.088892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:55.088892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:55.088892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:55.088892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:55.088892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:55.088892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:55.088892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:55.088892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:55.088892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:55.088892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:55.088892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:55.088892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:55.088892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:55.088892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:55.088892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:55.088892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:55.089814 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:55.089814 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:55.089814 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:55.089814 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:55.089814 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 317/633] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:55.090521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09385e-6485-4033-95ad-fb860499472e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:55.090814 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09385e-6485-4033-95ad-fb860499472e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:55.091096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09385e-6485-4033-95ad-fb860499472e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:55.092381 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 318/634] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:55.093422 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe8ba2-d348-4e6e-ad1f-cdcbaae225e9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:55.093714 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe8ba2-d348-4e6e-ad1f-cdcbaae225e9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:55.093994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe8ba2-d348-4e6e-ad1f-cdcbaae225e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:55.110166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-debe8ba2-d348-4e6e-ad1f-cdcbaae225e9 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:55.111351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-debe8ba2-d348-4e6e-ad1f-cdcbaae225e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:55.178333 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 319/635] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:55.179167 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe8ba2-d348-4e6e-ad1f-cdcbaae225e9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:55.181605 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:55.195069 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d7560-3bce-4716-9d12-86cf334d06fa None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:55.227068 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:55.230148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7726ef-41e7-4913-a011-8cc319808db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:55.230818 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7726ef-41e7-4913-a011-8cc319808db6 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:55.231105 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7726ef-41e7-4913-a011-8cc319808db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:55.231382 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7726ef-41e7-4913-a011-8cc319808db6 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:55.232194 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7726ef-41e7-4913-a011-8cc319808db6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:55.232505 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7726ef-41e7-4913-a011-8cc319808db6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:55.233949 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7726ef-41e7-4913-a011-8cc319808db6 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:55.246595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 317/636] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:55:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:55.247568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7726ef-41e7-4913-a011-8cc319808db6 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:55.285665 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe8ba2-d348-4e6e-ad1f-cdcbaae225e9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:55.318096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4915a-a347-4281-a4ff-b2c64f010320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:55.318888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4915a-a347-4281-a4ff-b2c64f010320 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:55.319187 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4915a-a347-4281-a4ff-b2c64f010320 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:55.319467 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4915a-a347-4281-a4ff-b2c64f010320 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:55.330279 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb4915a-a347-4281-a4ff-b2c64f010320 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:55.331785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb4915a-a347-4281-a4ff-b2c64f010320 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:55.344483 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7726ef-41e7-4913-a011-8cc319808db6 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:55.349491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ddb4915a-a347-4281-a4ff-b2c64f010320 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:55.349491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:55.349491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:55.349491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:55.349491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:55.349491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:55.349491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:55.349491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:55.349491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:55.349491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:55.349491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:55.349491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:55.349491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:55.349491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:55.349491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:55.350565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:55.350565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:55.350565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:55.350565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:55.350565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:55.350565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:55.350565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:55.350565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:55.350565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:55.350565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:55.350565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:55.350565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:55.350565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:55.350565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:55.350565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:55.350565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:55.350565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:55.351503 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:55.351503 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:55.351503 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:55.351503 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:55.351503 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 320/637] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:55.351871 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40737e-a2cf-4f41-80c6-a38488b4efcf None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:55.352208 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40737e-a2cf-4f41-80c6-a38488b4efcf None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:55.352489 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40737e-a2cf-4f41-80c6-a38488b4efcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:55.368253 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3c40737e-a2cf-4f41-80c6-a38488b4efcf None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:55.369587 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3c40737e-a2cf-4f41-80c6-a38488b4efcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:55.382316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf336e5f-d503-403a-b2bf-b55818df1945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:55.383473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf336e5f-d503-403a-b2bf-b55818df1945 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:55.383763 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf336e5f-d503-403a-b2bf-b55818df1945 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:55.384067 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf336e5f-d503-403a-b2bf-b55818df1945 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:55.384791 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf336e5f-d503-403a-b2bf-b55818df1945 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:55.385105 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf336e5f-d503-403a-b2bf-b55818df1945 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:55.386468 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf336e5f-d503-403a-b2bf-b55818df1945 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:55.397554 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 318/638] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:55.398400 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf336e5f-d503-403a-b2bf-b55818df1945 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:55.409371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:55.435587 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 321/639] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:55.436355 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40737e-a2cf-4f41-80c6-a38488b4efcf None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:55.462838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:55.505368 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf336e5f-d503-403a-b2bf-b55818df1945 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:55.535969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40737e-a2cf-4f41-80c6-a38488b4efcf None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:55.539113 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8121e21-c56c-4b3d-9cc6-1b06de054bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:55.540429 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8121e21-c56c-4b3d-9cc6-1b06de054bcb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:55.540715 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8121e21-c56c-4b3d-9cc6-1b06de054bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:55.540990 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8121e21-c56c-4b3d-9cc6-1b06de054bcb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:55.551129 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8121e21-c56c-4b3d-9cc6-1b06de054bcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:55.558749 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-b8121e21-c56c-4b3d-9cc6-1b06de054bcb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:55.558749 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:55.558749 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:55.558749 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:55.558749 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:55.558749 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:55.558749 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:55.558749 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:55.558749 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:55.558749 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:55.558749 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:55.558749 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:55.558749 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:55.558749 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:55.558749 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:55.558749 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:55.559819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:55.559819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:55.559819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:55.559819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:55.559819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:55.559819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:55.559819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:55.559819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:55.559819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:55.559819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:55.559819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:55.559819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:55.559819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:55.559819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:55.559819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:55.559819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:55.560708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:55.560708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:55.560708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:55.560708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:55.560708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:55.560708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:55.560708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:55.560708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:55.560708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:55.560708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:55.560708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:55.560708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:55.560708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 319/640] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:55:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:55.560708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8121e21-c56c-4b3d-9cc6-1b06de054bcb None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:55.577027 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c73339-4575-4822-a3fc-db7c414b1b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:55.577923 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c73339-4575-4822-a3fc-db7c414b1b2f None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:55.578213 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c73339-4575-4822-a3fc-db7c414b1b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:55.578612 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c73339-4575-4822-a3fc-db7c414b1b2f None admin] PATH_INFO: `/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/cbf513ee25374e49b3d8ce28843e393a/roles/5c4ba827ac854bc69cc26c3bcd27dc5c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:55.593379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c73339-4575-4822-a3fc-db7c414b1b2f None admin] RBAC: Authorizing `identity:create_grant(project_id=ec94946acf91486ba3f5b4fe834b8076, user_id=cbf513ee25374e49b3d8ce28843e393a, role_id=5c4ba827ac854bc69cc26c3bcd27dc5c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:55.594815 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c73339-4575-4822-a3fc-db7c414b1b2f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:55.608301 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 322/641] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:55:55 2022] PUT /identity/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/cbf513ee25374e49b3d8ce28843e393a/roles/5c4ba827ac854bc69cc26c3bcd27dc5c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:55:55.609093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c73339-4575-4822-a3fc-db7c414b1b2f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:55.645511 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:55.690169 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8121e21-c56c-4b3d-9cc6-1b06de054bcb None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:55.696827 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:55.724033 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c73339-4575-4822-a3fc-db7c414b1b2f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:55.731383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70670cd-33db-40c6-a234-1a6d8ea8b410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:55.732126 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70670cd-33db-40c6-a234-1a6d8ea8b410 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:55.732422 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70670cd-33db-40c6-a234-1a6d8ea8b410 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:55.732706 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70670cd-33db-40c6-a234-1a6d8ea8b410 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:55.733450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70670cd-33db-40c6-a234-1a6d8ea8b410 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:55.733763 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70670cd-33db-40c6-a234-1a6d8ea8b410 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:55.735182 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70670cd-33db-40c6-a234-1a6d8ea8b410 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:55.748330 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 320/642] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:55.749198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70670cd-33db-40c6-a234-1a6d8ea8b410 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:55.774879 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a64fd4-2ac7-4f80-ae3a-36bd3dbdbbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:55.775686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a64fd4-2ac7-4f80-ae3a-36bd3dbdbbc6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:55.776061 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a64fd4-2ac7-4f80-ae3a-36bd3dbdbbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:55.776291 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a64fd4-2ac7-4f80-ae3a-36bd3dbdbbc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:55.783164 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a64fd4-2ac7-4f80-ae3a-36bd3dbdbbc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:55.784687 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a64fd4-2ac7-4f80-ae3a-36bd3dbdbbc6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:55.798914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-80a64fd4-2ac7-4f80-ae3a-36bd3dbdbbc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:55.798914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:55.798914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:55.798914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:55.798914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:55.798914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:55.798914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:55.798914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:55.798914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:55.798914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:55.798914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:55.798914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:55.798914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:55:55.798914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:55:55.798914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:55:55.799888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:55:55.799888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:55.799888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:55.799888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:55.799888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:55.799888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:55.799888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:55.799888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:55.799888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:55.799888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:55.799888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:55.799888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:55.799888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:55.799888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:55.799888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:55.799888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:55.799888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:55.800818 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:55:55.800818 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:55:55.800818 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:55:55.800818 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:55:55.800818 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:55:55.800818 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:55:55.800818 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:55.800818 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:55.800818 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 323/643] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:55.801322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a64fd4-2ac7-4f80-ae3a-36bd3dbdbbc6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:55.866775 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70670cd-33db-40c6-a234-1a6d8ea8b410 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:55.894827 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:55.903208 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42c5d2-d6ac-4dce-bfb1-6fd6b5abf4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:55.904043 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42c5d2-d6ac-4dce-bfb1-6fd6b5abf4bf None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:55.904332 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42c5d2-d6ac-4dce-bfb1-6fd6b5abf4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:55.904611 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42c5d2-d6ac-4dce-bfb1-6fd6b5abf4bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:55.907319 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a64fd4-2ac7-4f80-ae3a-36bd3dbdbbc6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:55.915630 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b42c5d2-d6ac-4dce-bfb1-6fd6b5abf4bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:55.923300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-9b42c5d2-d6ac-4dce-bfb1-6fd6b5abf4bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:55.923300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:55.923300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:55.923300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:55.923300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:55.923300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:55.923300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:55.923300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:55.923300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:55.923300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:55.923300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:55.923300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:55.923300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:55.923300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:55.923300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:55.923300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:55.924303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:55.924303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:55.924303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:55.924303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:55.924303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:55.924303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:55.924303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:55.924303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:55.924303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:55.924303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:55.924303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:55.924303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:55.924303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:55.924303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:55.924303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:55.924303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:55.925198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:55.925198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:55.925198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:55.925198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:55.925198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:55.925198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:55.925198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:55.925198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:55.925198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:55.925198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:55.925198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:55.925198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:55.925198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 321/644] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:55:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:55.926000 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42c5d2-d6ac-4dce-bfb1-6fd6b5abf4bf None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:55.961865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101cfaf-defa-4f79-85e1-aa4fa05c5999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:55.962616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101cfaf-defa-4f79-85e1-aa4fa05c5999 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:55.963022 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101cfaf-defa-4f79-85e1-aa4fa05c5999 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:55.963295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101cfaf-defa-4f79-85e1-aa4fa05c5999 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:55.964139 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101cfaf-defa-4f79-85e1-aa4fa05c5999 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:55.964517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101cfaf-defa-4f79-85e1-aa4fa05c5999 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:55.966456 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101cfaf-defa-4f79-85e1-aa4fa05c5999 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:55.968519 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:55.980989 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 324/645] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Jul 18 13:55:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:55.980989 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101cfaf-defa-4f79-85e1-aa4fa05c5999 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:56.039743 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42c5d2-d6ac-4dce-bfb1-6fd6b5abf4bf None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:56.076076 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d27d57-09de-464b-9950-022c8984fd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:56.076873 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d27d57-09de-464b-9950-022c8984fd86 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:56.077157 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d27d57-09de-464b-9950-022c8984fd86 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:56.077622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d27d57-09de-464b-9950-022c8984fd86 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:56.085718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d27d57-09de-464b-9950-022c8984fd86 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:56.086544 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101cfaf-defa-4f79-85e1-aa4fa05c5999 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:56.087200 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d27d57-09de-464b-9950-022c8984fd86 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:56.102617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-82d27d57-09de-464b-9950-022c8984fd86 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 13:55:56.102617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:56.102617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:56.102617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:56.102617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:56.102617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:56.102617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:56.102617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:56.102617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:56.102617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:56.102617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:56.102617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:56.102617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:55:56.102617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:55:56.102617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:55:56.102617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:55:56.103618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:56.103618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:56.103618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:55:56.103618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:56.103618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:55:56.103618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:56.103618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:56.103618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:56.103618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:56.103618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:56.103618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:56.103618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:56.103618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:56.103618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:56.103618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:56.103618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:56.103618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:56.103618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:56.104949 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:56.104949 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:56.104949 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:55:56.104949 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:55:56.104949 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:55:56.104949 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:55:56.104949 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:55:56.104949 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:55:56.104949 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 13:55:56.104949 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:56.104949 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 322/646] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:56.104949 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d27d57-09de-464b-9950-022c8984fd86 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:56.135046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055ac0a-66fb-4710-a7ac-ce2ce0eb093f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:56.136740 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055ac0a-66fb-4710-a7ac-ce2ce0eb093f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:56.137459 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055ac0a-66fb-4710-a7ac-ce2ce0eb093f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:56.137894 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055ac0a-66fb-4710-a7ac-ce2ce0eb093f None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:56.138836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a055ac0a-66fb-4710-a7ac-ce2ce0eb093f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:56.139560 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a055ac0a-66fb-4710-a7ac-ce2ce0eb093f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:56.142074 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a055ac0a-66fb-4710-a7ac-ce2ce0eb093f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:56.155238 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:56.165086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 325/647] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:55:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:56.165086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055ac0a-66fb-4710-a7ac-ce2ce0eb093f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:56.214688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d27d57-09de-464b-9950-022c8984fd86 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:56.226126 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:56.248274 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074f4b8-5cd5-4e1f-ae7b-2b0caf6fe6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:56.249616 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074f4b8-5cd5-4e1f-ae7b-2b0caf6fe6fe None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:56.250011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074f4b8-5cd5-4e1f-ae7b-2b0caf6fe6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:56.250396 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074f4b8-5cd5-4e1f-ae7b-2b0caf6fe6fe None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:56.251611 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074f4b8-5cd5-4e1f-ae7b-2b0caf6fe6fe None admin] RBAC: Adding query filter params (name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:56.252054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074f4b8-5cd5-4e1f-ae7b-2b0caf6fe6fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:56.253786 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074f4b8-5cd5-4e1f-ae7b-2b0caf6fe6fe None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:56.262450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 323/648] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Jul 18 13:55:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:56.263152 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074f4b8-5cd5-4e1f-ae7b-2b0caf6fe6fe None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:56.278595 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055ac0a-66fb-4710-a7ac-ce2ce0eb093f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:56.312769 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21794108-f4ca-4e3e-b38c-644b1497de86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:56.313562 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21794108-f4ca-4e3e-b38c-644b1497de86 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:56.313845 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21794108-f4ca-4e3e-b38c-644b1497de86 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:56.314109 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21794108-f4ca-4e3e-b38c-644b1497de86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:56.324633 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21794108-f4ca-4e3e-b38c-644b1497de86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:56.326327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21794108-f4ca-4e3e-b38c-644b1497de86 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:56.343490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-21794108-f4ca-4e3e-b38c-644b1497de86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:56.343490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:56.343490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:56.343490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:56.343490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:56.343490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:56.343490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:56.343490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:56.343490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:56.343490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:56.343490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:56.343490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:56.343490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:56.343490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:56.343490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:56.344686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:56.344686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:56.344686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:56.344686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:56.344686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:56.344686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:56.344686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:56.344686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:56.344686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:56.344686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:56.344686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:56.344686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:56.344686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:56.344686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:56.344686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:56.344686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:56.344686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:56.345834 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:56.345834 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:56.345834 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:56.345834 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:56.360261 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 326/649] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:56.360261 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21794108-f4ca-4e3e-b38c-644b1497de86 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:56.387460 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074f4b8-5cd5-4e1f-ae7b-2b0caf6fe6fe None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:56.399988 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:56.425359 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4874fa4-4bc6-4706-89d1-4a09b4ad7c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:56.426124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4874fa4-4bc6-4706-89d1-4a09b4ad7c47 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:56.426504 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4874fa4-4bc6-4706-89d1-4a09b4ad7c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:56.426862 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4874fa4-4bc6-4706-89d1-4a09b4ad7c47 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:56.427805 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4874fa4-4bc6-4706-89d1-4a09b4ad7c47 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:56.428237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4874fa4-4bc6-4706-89d1-4a09b4ad7c47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:56.429738 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4874fa4-4bc6-4706-89d1-4a09b4ad7c47 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:56.452277 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 324/650] 10.222.0.94 () {56 vars in 1137 bytes} [Mon Jul 18 13:55:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:56.452277 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4874fa4-4bc6-4706-89d1-4a09b4ad7c47 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:56.476085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21794108-f4ca-4e3e-b38c-644b1497de86 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:56.487399 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:56.513416 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc3f52-8059-49a6-bca0-582223f89c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:56.514099 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc3f52-8059-49a6-bca0-582223f89c65 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:56.514451 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc3f52-8059-49a6-bca0-582223f89c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:56.514887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc3f52-8059-49a6-bca0-582223f89c65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:56.515696 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc3f52-8059-49a6-bca0-582223f89c65 None admin] RBAC: Adding query filter params (role.id=5c4ba827ac854bc69cc26c3bcd27dc5c, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=ac717fcd9fc1499db12cf852d45572a8) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:56.516118 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc3f52-8059-49a6-bca0-582223f89c65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:56.517622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc3f52-8059-49a6-bca0-582223f89c65 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:56.524186 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 327/651] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:55:56 2022] GET /identity/v3/role_assignments?user.id=ac717fcd9fc1499db12cf852d45572a8&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=5c4ba827ac854bc69cc26c3bcd27dc5c => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:56.524741 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc3f52-8059-49a6-bca0-582223f89c65 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:56.585337 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4874fa4-4bc6-4706-89d1-4a09b4ad7c47 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:56.627177 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c210d-e8d4-4162-9ea4-17c2aa066320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:56.628241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c210d-e8d4-4162-9ea4-17c2aa066320 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:56.628638 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c210d-e8d4-4162-9ea4-17c2aa066320 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:56.629162 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c210d-e8d4-4162-9ea4-17c2aa066320 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:56.642213 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c210d-e8d4-4162-9ea4-17c2aa066320 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:56.643895 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c210d-e8d4-4162-9ea4-17c2aa066320 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:56.655346 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc3f52-8059-49a6-bca0-582223f89c65 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:56.664607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-814c210d-e8d4-4162-9ea4-17c2aa066320 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:56.664607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:56.664607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:56.664607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:56.664607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:56.664607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:56.664607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:56.664607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:56.664607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:56.664607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:56.664607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:56.664607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:56.664607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:56.664607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:56.664607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:56.665761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:56.665761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:56.665761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:56.665761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:56.665761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:56.665761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:56.665761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:56.665761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:56.665761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:56.665761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:56.665761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:56.665761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:56.665761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:56.665761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:56.665761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:56.665761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:56.665761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:56.666877 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:56.666877 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:56.666877 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:56.666877 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:56.673008 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 325/652] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:56.674085 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c210d-e8d4-4162-9ea4-17c2aa066320 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:56.702607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a544ec0-300a-4e35-8e2b-879c1205bf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:56.703468 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:56.704142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a544ec0-300a-4e35-8e2b-879c1205bf27 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:56.704833 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a544ec0-300a-4e35-8e2b-879c1205bf27 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:56.705699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a544ec0-300a-4e35-8e2b-879c1205bf27 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:56.706831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a544ec0-300a-4e35-8e2b-879c1205bf27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:56.707466 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a544ec0-300a-4e35-8e2b-879c1205bf27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:56.709863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a544ec0-300a-4e35-8e2b-879c1205bf27 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:56.724849 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 328/653] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:56.725925 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a544ec0-300a-4e35-8e2b-879c1205bf27 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:56.770010 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:56.810321 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c210d-e8d4-4162-9ea4-17c2aa066320 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:56.848216 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c553e-8d06-452f-93a8-9bca7663b142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:56.849019 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c553e-8d06-452f-93a8-9bca7663b142 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:56.849457 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c553e-8d06-452f-93a8-9bca7663b142 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:56.849840 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c553e-8d06-452f-93a8-9bca7663b142 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:56.850893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a544ec0-300a-4e35-8e2b-879c1205bf27 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:56.895437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2d85d-c212-474d-bfb6-ea00f2c10ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:56.896398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2d85d-c212-474d-bfb6-ea00f2c10ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:56.897300 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2d85d-c212-474d-bfb6-ea00f2c10ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:56.897734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2d85d-c212-474d-bfb6-ea00f2c10ed3 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:56.899117 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2d85d-c212-474d-bfb6-ea00f2c10ed3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:56.899867 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2d85d-c212-474d-bfb6-ea00f2c10ed3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:56.901696 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2d85d-c212-474d-bfb6-ea00f2c10ed3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:56.916019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 329/654] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:56.918370 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2d85d-c212-474d-bfb6-ea00f2c10ed3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:56.949008 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c553e-8d06-452f-93a8-9bca7663b142 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:56.950968 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c553e-8d06-452f-93a8-9bca7663b142 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:56.974531 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:57.046080 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:57.054211 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2d85d-c212-474d-bfb6-ea00f2c10ed3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:57.080189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 326/655] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:57.080189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c553e-8d06-452f-93a8-9bca7663b142 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:57.104464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa00d8e-3355-4e1e-b2bc-89246234a4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:57.106271 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa00d8e-3355-4e1e-b2bc-89246234a4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:57.107056 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa00d8e-3355-4e1e-b2bc-89246234a4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:57.107570 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa00d8e-3355-4e1e-b2bc-89246234a4d1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:57.118256 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa00d8e-3355-4e1e-b2bc-89246234a4d1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:57.119119 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa00d8e-3355-4e1e-b2bc-89246234a4d1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:57.133626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-0fa00d8e-3355-4e1e-b2bc-89246234a4d1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 13:55:57.133626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:57.133626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:57.133626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:57.133626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:57.133626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:57.133626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:57.133626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:57.133626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:57.133626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:57.133626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:57.133626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:57.133626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:55:57.133626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:55:57.133626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:55:57.133626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:55:57.134631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:57.134631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:57.134631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:55:57.134631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:57.134631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:55:57.134631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:57.134631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:57.134631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:57.134631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:57.134631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:57.134631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:57.134631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:57.134631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:57.134631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:57.134631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:57.134631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:57.134631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:57.134631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:57.135603 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:57.135603 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:57.135603 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:55:57.135603 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:55:57.135603 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:55:57.135603 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:55:57.135603 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:55:57.135603 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:55:57.135603 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 13:55:57.135603 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:57.135603 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 330/656] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:57.136262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa00d8e-3355-4e1e-b2bc-89246234a4d1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:57.201317 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c553e-8d06-452f-93a8-9bca7663b142 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:57.229071 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:57.250130 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abddaed-239d-4f01-93ac-5e6cc4cd7665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:57.251792 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abddaed-239d-4f01-93ac-5e6cc4cd7665 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:57.252418 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abddaed-239d-4f01-93ac-5e6cc4cd7665 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:57.252885 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abddaed-239d-4f01-93ac-5e6cc4cd7665 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:57.254132 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa00d8e-3355-4e1e-b2bc-89246234a4d1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:57.262423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abddaed-239d-4f01-93ac-5e6cc4cd7665 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:57.264443 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abddaed-239d-4f01-93ac-5e6cc4cd7665 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:57.283063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-6abddaed-239d-4f01-93ac-5e6cc4cd7665 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 13:55:57.283063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:57.283063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:57.283063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:57.283063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:57.283063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:57.283063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:57.283063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:57.283063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:57.283063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:57.283063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:57.283063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:57.283063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:55:57.283063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:55:57.283063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:55:57.283063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:55:57.284088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:57.284088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:57.284088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:55:57.284088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:57.284088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:55:57.284088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:57.284088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:57.284088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:57.284088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:57.284088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:57.284088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:57.284088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:57.284088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:57.284088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:57.284088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:57.284088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:57.284088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:57.284088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:57.285040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:57.285040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:57.285040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:55:57.285040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:55:57.285040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:55:57.285040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:55:57.285040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:55:57.285040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:55:57.285040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 13:55:57.285040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:57.289073 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 327/657] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:55:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:57.289834 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abddaed-239d-4f01-93ac-5e6cc4cd7665 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:57.300005 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb4afa-05ec-461b-834b-10f989b157f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:57.301571 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb4afa-05ec-461b-834b-10f989b157f3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:57.302081 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb4afa-05ec-461b-834b-10f989b157f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:57.302956 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb4afa-05ec-461b-834b-10f989b157f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:57.318134 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb4afa-05ec-461b-834b-10f989b157f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:57.321450 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb4afa-05ec-461b-834b-10f989b157f3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:57.345147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f3eb4afa-05ec-461b-834b-10f989b157f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:57.345147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:57.345147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:57.345147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:57.345147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:57.345147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:57.345147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:57.345147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:57.345147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:57.345147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:57.345147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:57.345147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:57.345147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:57.345147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:57.345147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:57.346104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:57.346104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:57.346104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:57.346104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:57.346104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:57.346104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:57.346104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:57.346104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:57.346104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:57.346104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:57.346104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:57.346104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:57.346104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:57.346104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:57.346104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:57.346104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:57.346104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:57.347010 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:57.347010 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:57.347010 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:57.347010 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:57.354634 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 331/658] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:57.356122 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb4afa-05ec-461b-834b-10f989b157f3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:57.368905 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:57.405037 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abddaed-239d-4f01-93ac-5e6cc4cd7665 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:57.447332 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab29769-ae59-4139-83a0-b15a5991c2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:57.448619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab29769-ae59-4139-83a0-b15a5991c2b0 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:57.449092 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab29769-ae59-4139-83a0-b15a5991c2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:57.449527 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab29769-ae59-4139-83a0-b15a5991c2b0 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:57.450783 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab29769-ae59-4139-83a0-b15a5991c2b0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:57.452135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab29769-ae59-4139-83a0-b15a5991c2b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:57.453804 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab29769-ae59-4139-83a0-b15a5991c2b0 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:57.479319 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 328/659] 10.222.0.94 () {56 vars in 1137 bytes} [Mon Jul 18 13:55:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:57.480859 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab29769-ae59-4139-83a0-b15a5991c2b0 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:57.483589 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb4afa-05ec-461b-834b-10f989b157f3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:57.519678 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae8b5d-e77e-44ed-888e-29949abed3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:57.520528 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae8b5d-e77e-44ed-888e-29949abed3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:57.520917 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae8b5d-e77e-44ed-888e-29949abed3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:57.521298 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae8b5d-e77e-44ed-888e-29949abed3e2 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:57.522277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae8b5d-e77e-44ed-888e-29949abed3e2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:57.522668 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae8b5d-e77e-44ed-888e-29949abed3e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:57.524307 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae8b5d-e77e-44ed-888e-29949abed3e2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:57.535700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:57.548092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 332/660] 10.222.0.94 () {56 vars in 1139 bytes} [Mon Jul 18 13:55:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:57.549171 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae8b5d-e77e-44ed-888e-29949abed3e2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:57.617198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab29769-ae59-4139-83a0-b15a5991c2b0 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:57.619202 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:57.658674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d1a84-7c3c-461f-8265-5ebb69abe1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:57.659335 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d1a84-7c3c-461f-8265-5ebb69abe1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:57.659624 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d1a84-7c3c-461f-8265-5ebb69abe1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:57.659905 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d1a84-7c3c-461f-8265-5ebb69abe1f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:57.660654 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d1a84-7c3c-461f-8265-5ebb69abe1f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:57.660965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d1a84-7c3c-461f-8265-5ebb69abe1f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:57.661160 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae8b5d-e77e-44ed-888e-29949abed3e2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:57.662376 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d1a84-7c3c-461f-8265-5ebb69abe1f4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:57.678234 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 329/661] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:57.678234 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d1a84-7c3c-461f-8265-5ebb69abe1f4 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:57.701480 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddd23f-1ff0-4180-b46c-6dd3c9b158a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:57.702293 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddd23f-1ff0-4180-b46c-6dd3c9b158a9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:57.702695 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddd23f-1ff0-4180-b46c-6dd3c9b158a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:57.702980 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddd23f-1ff0-4180-b46c-6dd3c9b158a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:57.773914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dddd23f-1ff0-4180-b46c-6dd3c9b158a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:57.775428 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dddd23f-1ff0-4180-b46c-6dd3c9b158a9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:57.779645 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:57.786336 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d1a84-7c3c-461f-8265-5ebb69abe1f4 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:57.827157 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4baaa0-5e25-4d65-bd03-58e980c4777a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:57.827862 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4baaa0-5e25-4d65-bd03-58e980c4777a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:57.828178 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4baaa0-5e25-4d65-bd03-58e980c4777a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:57.828457 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4baaa0-5e25-4d65-bd03-58e980c4777a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:57.842630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:57.885990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 333/662] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:57.887384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fcc72-bb9d-4532-ad48-c798781196cb None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:57.887681 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fcc72-bb9d-4532-ad48-c798781196cb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:57.887996 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fcc72-bb9d-4532-ad48-c798781196cb None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:57.889394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 334/663] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:57.889995 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135fcc72-bb9d-4532-ad48-c798781196cb None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:57.907799 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4baaa0-5e25-4d65-bd03-58e980c4777a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:57.909326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4baaa0-5e25-4d65-bd03-58e980c4777a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:58.006714 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135fcc72-bb9d-4532-ad48-c798781196cb None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:58.009703 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:58.019350 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 330/664] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:55:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:55:58.020301 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4baaa0-5e25-4d65-bd03-58e980c4777a None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:58.048230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd8ae9-305f-4381-88ca-4c370fb2da3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:58.049031 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd8ae9-305f-4381-88ca-4c370fb2da3b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:58.049315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd8ae9-305f-4381-88ca-4c370fb2da3b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:58.049593 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd8ae9-305f-4381-88ca-4c370fb2da3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:58.056305 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd8ae9-305f-4381-88ca-4c370fb2da3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:58.057765 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd8ae9-305f-4381-88ca-4c370fb2da3b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:58.071694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6afd8ae9-305f-4381-88ca-4c370fb2da3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:58.071694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:58.071694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:58.071694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:58.071694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:58.071694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:58.071694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:58.071694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:58.071694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:58.071694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:58.071694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:58.071694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:58.071694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:55:58.071694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:55:58.071694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:55:58.072731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:55:58.072731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:58.072731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:58.072731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:58.072731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:58.072731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:58.072731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:58.072731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:58.072731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:58.072731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:58.072731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:58.072731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:58.072731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:58.072731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:58.072731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:58.072731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:58.072731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:58.073841 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:55:58.073841 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:55:58.073841 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:55:58.073841 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:55:58.073841 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:55:58.073841 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:55:58.073841 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:58.073841 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:58.077351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 335/665] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:58.078571 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c4798-20fc-4fae-8bf8-63ed4fa274fb None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:58.078866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c4798-20fc-4fae-8bf8-63ed4fa274fb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:58.079146 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c4798-20fc-4fae-8bf8-63ed4fa274fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:58.096371 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6f2c4798-20fc-4fae-8bf8-63ed4fa274fb None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:58.097564 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6f2c4798-20fc-4fae-8bf8-63ed4fa274fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:58.115467 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:58.131073 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4baaa0-5e25-4d65-bd03-58e980c4777a None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:58.167493 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 336/666] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:58.168992 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c4798-20fc-4fae-8bf8-63ed4fa274fb None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:58.169939 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cbd084-42c9-4ec6-98e5-b06d995ac690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:58.170799 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cbd084-42c9-4ec6-98e5-b06d995ac690 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:58.171156 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cbd084-42c9-4ec6-98e5-b06d995ac690 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:58.171507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cbd084-42c9-4ec6-98e5-b06d995ac690 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:58.191757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cbd084-42c9-4ec6-98e5-b06d995ac690 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:58.193470 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cbd084-42c9-4ec6-98e5-b06d995ac690 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:58.218114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-83cbd084-42c9-4ec6-98e5-b06d995ac690 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:58.218114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:58.218114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:58.218114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:58.218114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:58.218114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:58.218114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:58.218114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:58.218114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:58.218114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:58.218114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:58.218114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:58.218114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:58.218114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:58.218114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:58.219203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:58.219203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:58.219203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:58.219203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:58.219203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:58.219203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:58.219203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:58.219203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:58.219203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:58.219203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:58.219203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:58.219203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:58.219203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:58.219203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:58.219203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:58.219203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:58.219203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:58.220328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:58.220328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:58.220328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:58.220328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:58.226007 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 331/667] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:58.226924 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cbd084-42c9-4ec6-98e5-b06d995ac690 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:58.278664 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:58.281835 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c4798-20fc-4fae-8bf8-63ed4fa274fb None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:58.317661 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a66ca1-c5aa-4602-b115-d9586de6d45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:58.318455 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a66ca1-c5aa-4602-b115-d9586de6d45b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:58.318743 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a66ca1-c5aa-4602-b115-d9586de6d45b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:58.319024 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a66ca1-c5aa-4602-b115-d9586de6d45b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:58.330366 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a66ca1-c5aa-4602-b115-d9586de6d45b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:58.331298 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cbd084-42c9-4ec6-98e5-b06d995ac690 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:58.331847 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a66ca1-c5aa-4602-b115-d9586de6d45b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:58.350727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b0a66ca1-c5aa-4602-b115-d9586de6d45b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:58.350727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:58.350727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:58.350727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:58.350727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:58.350727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:58.350727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:58.350727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:58.350727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:58.350727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:58.350727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:58.350727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:58.350727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:58.350727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:58.350727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:58.351683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:58.351683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:58.351683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:58.351683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:58.351683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:58.351683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:58.351683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:58.351683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:58.351683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:58.351683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:58.351683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:58.351683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:58.351683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:58.351683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:58.351683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:58.351683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:58.351683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:58.357596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:58.357596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:58.357596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:58.357596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:58.357596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 337/668] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:58.357596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a66ca1-c5aa-4602-b115-d9586de6d45b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:58.364438 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:58.374814 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2010a93-4e22-45b5-827b-2cadbedcd1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:58.375494 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2010a93-4e22-45b5-827b-2cadbedcd1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:58.375784 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2010a93-4e22-45b5-827b-2cadbedcd1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:58.376144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2010a93-4e22-45b5-827b-2cadbedcd1c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:58.376945 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2010a93-4e22-45b5-827b-2cadbedcd1c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:58.377261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2010a93-4e22-45b5-827b-2cadbedcd1c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:58.378705 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2010a93-4e22-45b5-827b-2cadbedcd1c6 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:58.396351 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 332/669] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:55:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:58.396351 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2010a93-4e22-45b5-827b-2cadbedcd1c6 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:58.468451 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a66ca1-c5aa-4602-b115-d9586de6d45b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:58.495563 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2010a93-4e22-45b5-827b-2cadbedcd1c6 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:58.503848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238989d1-8f48-4647-8821-3bbe17141be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:58.504776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238989d1-8f48-4647-8821-3bbe17141be3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:58.505068 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238989d1-8f48-4647-8821-3bbe17141be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:58.505350 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238989d1-8f48-4647-8821-3bbe17141be3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:58.505703 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:58.517757 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238989d1-8f48-4647-8821-3bbe17141be3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:58.526264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-238989d1-8f48-4647-8821-3bbe17141be3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:58.526264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:58.526264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:58.526264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:58.526264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:58.526264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:58.526264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:58.526264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:58.526264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:58.526264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:58.526264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:58.526264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:58.526264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:58.526264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:58.526264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:58.526264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:58.527194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:58.527194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:58.527194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:58.527194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:58.527194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:58.527194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:58.527194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:58.527194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:58.527194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:58.527194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:58.527194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:58.527194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:58.527194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:58.527194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:58.527194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:58.527194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:58.528522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:58.528522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:58.528522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:58.528522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:58.528522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:58.528522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:58.528522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:58.528522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:58.528522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:58.528522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:58.528522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:55:58.528522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:58.528522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 338/670] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:55:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:58.528522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238989d1-8f48-4647-8821-3bbe17141be3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:58.535669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301b086-2815-4da9-af09-4d4483dfc586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:58.536381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301b086-2815-4da9-af09-4d4483dfc586 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:58.536672 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301b086-2815-4da9-af09-4d4483dfc586 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:58.537082 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301b086-2815-4da9-af09-4d4483dfc586 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:58.537816 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f301b086-2815-4da9-af09-4d4483dfc586 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:58.538136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f301b086-2815-4da9-af09-4d4483dfc586 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:58.539642 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f301b086-2815-4da9-af09-4d4483dfc586 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:58.551671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 333/671] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:58.552518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301b086-2815-4da9-af09-4d4483dfc586 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:58.604472 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:58.642072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238989d1-8f48-4647-8821-3bbe17141be3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:58.674752 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301b086-2815-4da9-af09-4d4483dfc586 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:58.680919 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf554046-4ae5-434f-9051-ffa5e2f61a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:58.681871 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf554046-4ae5-434f-9051-ffa5e2f61a4d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:58.682466 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf554046-4ae5-434f-9051-ffa5e2f61a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:58.682799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf554046-4ae5-434f-9051-ffa5e2f61a4d None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:58.683641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf554046-4ae5-434f-9051-ffa5e2f61a4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:58.684007 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf554046-4ae5-434f-9051-ffa5e2f61a4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:58.685486 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf554046-4ae5-434f-9051-ffa5e2f61a4d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:58.698405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 339/672] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:58.699348 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf554046-4ae5-434f-9051-ffa5e2f61a4d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:58.720036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7b7b2-c864-4afa-880b-18dfc9bf6649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:58.720645 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7b7b2-c864-4afa-880b-18dfc9bf6649 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:58.720936 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7b7b2-c864-4afa-880b-18dfc9bf6649 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:58.721217 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7b7b2-c864-4afa-880b-18dfc9bf6649 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:58.722054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7b7b2-c864-4afa-880b-18dfc9bf6649 None admin] RBAC: Adding query filter params (role.id=5c4ba827ac854bc69cc26c3bcd27dc5c, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=503a8228d4754c309fc3f365f4cfc61b) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:58.722376 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7b7b2-c864-4afa-880b-18dfc9bf6649 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:58.723827 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7b7b2-c864-4afa-880b-18dfc9bf6649 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:58.743973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 334/673] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:55:58 2022] GET /identity/v3/role_assignments?user.id=503a8228d4754c309fc3f365f4cfc61b&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=5c4ba827ac854bc69cc26c3bcd27dc5c => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:58.743973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7b7b2-c864-4afa-880b-18dfc9bf6649 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:58.769877 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:58.824567 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf554046-4ae5-434f-9051-ffa5e2f61a4d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:58.848317 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7b7b2-c864-4afa-880b-18dfc9bf6649 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:58.854825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:58.866222 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6ec2b0-77d4-41b1-818e-1b5200e60e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:58.867793 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ec2b0-77d4-41b1-818e-1b5200e60e24 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:58.868184 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ec2b0-77d4-41b1-818e-1b5200e60e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:58.868473 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6ec2b0-77d4-41b1-818e-1b5200e60e24 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:58.869239 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ec2b0-77d4-41b1-818e-1b5200e60e24 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:58.869561 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ec2b0-77d4-41b1-818e-1b5200e60e24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:58.870994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6ec2b0-77d4-41b1-818e-1b5200e60e24 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:58.880163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 340/674] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Jul 18 13:55:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:58.881480 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2da64-b630-4905-81ff-7e740660850e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:58.881480 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2da64-b630-4905-81ff-7e740660850e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:58.881480 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2da64-b630-4905-81ff-7e740660850e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:58.884041 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 341/675] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:55:58.884041 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2da64-b630-4905-81ff-7e740660850e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:58.899080 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0609b54-95c0-46fa-9139-d9a1533024d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:58.899920 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0609b54-95c0-46fa-9139-d9a1533024d2 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:58.900240 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0609b54-95c0-46fa-9139-d9a1533024d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:58.900518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0609b54-95c0-46fa-9139-d9a1533024d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:58.907548 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0609b54-95c0-46fa-9139-d9a1533024d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:58.909065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0609b54-95c0-46fa-9139-d9a1533024d2 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:58.926435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-c0609b54-95c0-46fa-9139-d9a1533024d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:58.926435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:58.926435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:58.926435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:58.926435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:58.926435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:58.926435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:58.926435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:58.926435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:58.926435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:58.926435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:58.926435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:58.926435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:55:58.926435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:55:58.926435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:55:58.927449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:55:58.927449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:58.927449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:58.927449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:58.927449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:58.927449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:58.927449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:58.927449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:58.927449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:58.927449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:58.927449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:58.927449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:58.927449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:58.927449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:58.927449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:58.927449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:58.927449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:58.928490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:55:58.928490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:55:58.928490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:55:58.928490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:55:58.928490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:55:58.928490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:55:58.928490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:58.928490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:58.928490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 335/676] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:58.933182 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0609b54-95c0-46fa-9139-d9a1533024d2 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:59.016193 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2da64-b630-4905-81ff-7e740660850e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:59.042615 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:59.050880 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0609b54-95c0-46fa-9139-d9a1533024d2 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:59.052350 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd5ab5-ed83-4534-bae6-493915a73981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:59.053156 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd5ab5-ed83-4534-bae6-493915a73981 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:59.053442 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd5ab5-ed83-4534-bae6-493915a73981 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:59.053718 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd5ab5-ed83-4534-bae6-493915a73981 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:59.060860 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd5ab5-ed83-4534-bae6-493915a73981 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:59.062303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd5ab5-ed83-4534-bae6-493915a73981 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:59.077116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-bbfd5ab5-ed83-4534-bae6-493915a73981 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:59.077116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:59.077116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:59.077116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:59.077116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:59.077116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:59.077116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:59.077116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:59.077116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:59.077116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:59.077116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:59.077116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:59.077116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:55:59.077116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:55:59.077116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:55:59.078200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:55:59.078200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:59.078200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:59.078200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:59.078200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:59.078200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:59.078200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:59.078200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:59.078200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:59.078200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:59.078200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:59.078200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:59.078200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:59.078200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:59.078200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:59.078200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:59.078200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:59.079680 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:55:59.079680 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:55:59.079680 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:55:59.079680 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:55:59.079680 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:55:59.079680 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:55:59.079680 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:55:59.079680 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:59.083260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 342/677] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:55:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:59.084546 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97840406-1c13-41b8-8089-4f6f0415584e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:59.084844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97840406-1c13-41b8-8089-4f6f0415584e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:59.085125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97840406-1c13-41b8-8089-4f6f0415584e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:59.090667 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29790ad-34a0-4099-92fc-a9db7f7553fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:59.091661 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29790ad-34a0-4099-92fc-a9db7f7553fc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:59.091784 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29790ad-34a0-4099-92fc-a9db7f7553fc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:59.092044 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29790ad-34a0-4099-92fc-a9db7f7553fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:59.102561 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-97840406-1c13-41b8-8089-4f6f0415584e None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:55:59.103188 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29790ad-34a0-4099-92fc-a9db7f7553fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:59.103813 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-97840406-1c13-41b8-8089-4f6f0415584e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:55:59.104689 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29790ad-34a0-4099-92fc-a9db7f7553fc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:59.123547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-e29790ad-34a0-4099-92fc-a9db7f7553fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:59.123547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:59.123547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:59.123547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:59.123547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:59.123547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:59.123547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:59.123547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:59.123547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:59.123547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:59.123547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:59.123547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:59.123547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:55:59.123547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:55:59.123547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:55:59.124710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:55:59.124710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:59.124710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:59.124710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:59.124710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:59.124710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:59.124710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:59.124710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:59.124710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:59.124710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:59.124710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:59.124710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:59.124710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:59.124710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:59.124710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:59.124710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:59.124710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:59.125637 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:55:59.125637 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:55:59.125637 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:55:59.125637 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:59.129482 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 336/678] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:59.130140 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29790ad-34a0-4099-92fc-a9db7f7553fc None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:59.150846 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:59.180154 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 343/679] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:55:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:55:59.180154 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97840406-1c13-41b8-8089-4f6f0415584e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:59.237992 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29790ad-34a0-4099-92fc-a9db7f7553fc None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:59.272161 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d1ac2-b318-423f-a907-8f5d780f6449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:59.272752 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d1ac2-b318-423f-a907-8f5d780f6449 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:59.273042 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d1ac2-b318-423f-a907-8f5d780f6449 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:59.273850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d1ac2-b318-423f-a907-8f5d780f6449 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:59.274645 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d1ac2-b318-423f-a907-8f5d780f6449 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:59.274962 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d1ac2-b318-423f-a907-8f5d780f6449 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:59.276430 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d1ac2-b318-423f-a907-8f5d780f6449 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:59.293440 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 337/680] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:55:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:59.293440 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d1ac2-b318-423f-a907-8f5d780f6449 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:59.300959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:59.303728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97840406-1c13-41b8-8089-4f6f0415584e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:59.339847 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0fe97-03bc-43c5-bc96-72e74b683d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:59.340565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0fe97-03bc-43c5-bc96-72e74b683d98 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:59.340855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0fe97-03bc-43c5-bc96-72e74b683d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:59.341137 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0fe97-03bc-43c5-bc96-72e74b683d98 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:59.341935 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0fe97-03bc-43c5-bc96-72e74b683d98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:59.342254 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0fe97-03bc-43c5-bc96-72e74b683d98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:59.343705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0fe97-03bc-43c5-bc96-72e74b683d98 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:59.357325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 344/681] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:55:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:59.358102 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0fe97-03bc-43c5-bc96-72e74b683d98 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:59.399689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:59.412329 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d1ac2-b318-423f-a907-8f5d780f6449 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:59.448553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293fb49-d297-4c32-b17e-918d308e2b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:59.449381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293fb49-d297-4c32-b17e-918d308e2b36 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:59.449770 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293fb49-d297-4c32-b17e-918d308e2b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:59.450148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293fb49-d297-4c32-b17e-918d308e2b36 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:59.450972 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9293fb49-d297-4c32-b17e-918d308e2b36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:59.451377 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9293fb49-d297-4c32-b17e-918d308e2b36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:59.452982 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9293fb49-d297-4c32-b17e-918d308e2b36 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:59.464557 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0fe97-03bc-43c5-bc96-72e74b683d98 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:59.473333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 338/682] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:59.473333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293fb49-d297-4c32-b17e-918d308e2b36 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:59.499294 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65903383-88db-4ec6-92be-0f39c68752bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:59.500133 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65903383-88db-4ec6-92be-0f39c68752bd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:59.500422 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65903383-88db-4ec6-92be-0f39c68752bd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:59.500704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65903383-88db-4ec6-92be-0f39c68752bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:59.511674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65903383-88db-4ec6-92be-0f39c68752bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:55:59.519325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-65903383-88db-4ec6-92be-0f39c68752bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:55:59.519325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:59.519325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:59.519325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:59.519325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:59.519325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:59.519325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:59.519325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:59.519325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:59.519325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:59.519325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:59.519325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:59.519325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:55:59.519325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:55:59.519325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:55:59.519325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:55:59.520452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:55:59.520452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:55:59.520452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:55:59.520452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:55:59.520452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:59.520452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:59.520452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:59.520452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:59.520452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:59.520452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:59.520452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:59.520452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:59.520452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:59.520452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:59.520452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:59.520452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:59.521512 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:59.521512 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:59.521512 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:59.521512 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:59.521512 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:55:59.521512 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:55:59.521512 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:55:59.521512 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:55:59.521512 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:55:59.521512 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:55:59.521512 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:55:59.521512 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:55:59.525586 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 345/683] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:55:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:59.526179 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65903383-88db-4ec6-92be-0f39c68752bd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:59.550157 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:59.596307 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293fb49-d297-4c32-b17e-918d308e2b36 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:59.633264 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda4b63-4e1b-4837-9e49-7d2afe902fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:59.634052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda4b63-4e1b-4837-9e49-7d2afe902fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:59.634409 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda4b63-4e1b-4837-9e49-7d2afe902fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:59.635014 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda4b63-4e1b-4837-9e49-7d2afe902fa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:59.635886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda4b63-4e1b-4837-9e49-7d2afe902fa2 None admin] RBAC: Adding query filter params (role.id=5c4ba827ac854bc69cc26c3bcd27dc5c, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=45ee15f7716541a1a61443069d5cf4f8) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:59.636238 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda4b63-4e1b-4837-9e49-7d2afe902fa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:59.637721 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda4b63-4e1b-4837-9e49-7d2afe902fa2 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:59.637847 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:59.642256 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65903383-88db-4ec6-92be-0f39c68752bd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:59.644309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 339/684] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:55:59 2022] GET /identity/v3/role_assignments?user.id=45ee15f7716541a1a61443069d5cf4f8&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=5c4ba827ac854bc69cc26c3bcd27dc5c => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:59.645086 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda4b63-4e1b-4837-9e49-7d2afe902fa2 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:59.698379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94986513-1de4-4b0f-a35d-f81dd637836e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:59.700061 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94986513-1de4-4b0f-a35d-f81dd637836e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:59.700531 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94986513-1de4-4b0f-a35d-f81dd637836e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:59.700989 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94986513-1de4-4b0f-a35d-f81dd637836e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:59.702591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94986513-1de4-4b0f-a35d-f81dd637836e None admin] RBAC: Adding query filter params (role.id=5c4ba827ac854bc69cc26c3bcd27dc5c, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=14027a0e13fd4272bd7803e3131b6348) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:59.703102 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94986513-1de4-4b0f-a35d-f81dd637836e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:59.705898 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94986513-1de4-4b0f-a35d-f81dd637836e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:59.721158 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 346/685] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:55:59 2022] GET /identity/v3/role_assignments?user.id=14027a0e13fd4272bd7803e3131b6348&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=5c4ba827ac854bc69cc26c3bcd27dc5c => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:59.721158 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94986513-1de4-4b0f-a35d-f81dd637836e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:59.821273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda4b63-4e1b-4837-9e49-7d2afe902fa2 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:59.857052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:55:59.867338 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff72396-91a9-42a2-aa44-4877b2fd5352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:59.868211 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff72396-91a9-42a2-aa44-4877b2fd5352 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:59.868689 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff72396-91a9-42a2-aa44-4877b2fd5352 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:59.869040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff72396-91a9-42a2-aa44-4877b2fd5352 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:59.877597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff72396-91a9-42a2-aa44-4877b2fd5352 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:59.879170 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff72396-91a9-42a2-aa44-4877b2fd5352 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:59.899065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-7ff72396-91a9-42a2-aa44-4877b2fd5352 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 13:55:59.899065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:55:59.899065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:55:59.899065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:55:59.899065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:55:59.899065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:55:59.899065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:55:59.899065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:55:59.899065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:55:59.899065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:55:59.899065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:55:59.899065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:55:59.899065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:55:59.899065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:55:59.899065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:55:59.899065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:55:59.900142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:55:59.900142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:55:59.900142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:55:59.900142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:59.900142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:55:59.900142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:55:59.900142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:55:59.900142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:55:59.900142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:55:59.900142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:55:59.900142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:55:59.900142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:55:59.900142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:55:59.900142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:55:59.900142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:55:59.900142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:55:59.900142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:55:59.900142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:55:59.901148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:55:59.901148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:55:59.901148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:55:59.901148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:55:59.901148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:55:59.901148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:55:59.901148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:55:59.901148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:55:59.901148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 13:55:59.901148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:55:59.901715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94986513-1de4-4b0f-a35d-f81dd637836e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:55:59.901791 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 340/686] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:55:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:55:59.908368 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff72396-91a9-42a2-aa44-4877b2fd5352 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:55:59.951581 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7713f-8406-44b2-b36e-048b3c00be8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:55:59.951581 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7713f-8406-44b2-b36e-048b3c00be8c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:55:59.951581 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7713f-8406-44b2-b36e-048b3c00be8c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:55:59.951581 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7713f-8406-44b2-b36e-048b3c00be8c None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:55:59.951900 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7713f-8406-44b2-b36e-048b3c00be8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:55:59.952700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7713f-8406-44b2-b36e-048b3c00be8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:55:59.955706 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7713f-8406-44b2-b36e-048b3c00be8c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:55:59.968694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 347/687] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:55:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:55:59.972403 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7713f-8406-44b2-b36e-048b3c00be8c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:00.008762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:00.040241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff72396-91a9-42a2-aa44-4877b2fd5352 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:00.081940 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be496442-6f99-4f5c-82c5-51055dfe1f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:00.082560 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be496442-6f99-4f5c-82c5-51055dfe1f58 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:00.082793 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be496442-6f99-4f5c-82c5-51055dfe1f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:00.083072 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be496442-6f99-4f5c-82c5-51055dfe1f58 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:00.083913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be496442-6f99-4f5c-82c5-51055dfe1f58 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:00.085465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be496442-6f99-4f5c-82c5-51055dfe1f58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:00.087329 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be496442-6f99-4f5c-82c5-51055dfe1f58 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:00.109034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7713f-8406-44b2-b36e-048b3c00be8c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:00.109679 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 341/688] 10.222.0.94 () {56 vars in 1135 bytes} [Mon Jul 18 13:55:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:00.117643 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be496442-6f99-4f5c-82c5-51055dfe1f58 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:00.152396 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e79ca-c36d-4991-aa7d-75b823964caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:00.153419 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e79ca-c36d-4991-aa7d-75b823964caf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:00.153801 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e79ca-c36d-4991-aa7d-75b823964caf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:00.154179 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e79ca-c36d-4991-aa7d-75b823964caf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:00.165988 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e79ca-c36d-4991-aa7d-75b823964caf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:00.167603 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e79ca-c36d-4991-aa7d-75b823964caf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:00.183718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:00.185327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-953e79ca-c36d-4991-aa7d-75b823964caf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:00.185327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:00.185327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:00.185327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:00.185327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:00.185327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:00.185327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:00.185327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:00.185327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:00.185327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:00.185327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:00.185327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:00.185327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:00.185327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:00.185327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:00.186418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:00.186418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:00.186418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:00.186418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:00.186418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:00.186418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:00.186418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:00.186418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:00.186418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:00.186418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:00.186418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:00.186418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:00.186418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:00.186418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:00.186418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:00.186418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:00.186418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:00.187589 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:00.187589 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:00.187589 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:00.187589 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:00.192073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 348/689] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:55:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:00.194164 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e79ca-c36d-4991-aa7d-75b823964caf None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:00.256488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be496442-6f99-4f5c-82c5-51055dfe1f58 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:00.276098 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:00.299758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e79ca-c36d-4991-aa7d-75b823964caf None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:00.303223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371626ca-e932-49e4-9259-a6c33b0ac328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:00.304114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371626ca-e932-49e4-9259-a6c33b0ac328 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:00.304590 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371626ca-e932-49e4-9259-a6c33b0ac328 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:00.305023 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371626ca-e932-49e4-9259-a6c33b0ac328 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:00.364368 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bcae9-68ff-426b-8058-7a516fcf100b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:00.365833 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bcae9-68ff-426b-8058-7a516fcf100b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:00.366122 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bcae9-68ff-426b-8058-7a516fcf100b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:00.366398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bcae9-68ff-426b-8058-7a516fcf100b None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:00.367239 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269bcae9-68ff-426b-8058-7a516fcf100b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:00.367546 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269bcae9-68ff-426b-8058-7a516fcf100b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:00.368951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269bcae9-68ff-426b-8058-7a516fcf100b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:00.389381 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 349/690] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:00.395687 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bcae9-68ff-426b-8058-7a516fcf100b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:00.424228 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371626ca-e932-49e4-9259-a6c33b0ac328 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:00.425861 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371626ca-e932-49e4-9259-a6c33b0ac328 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:00.481260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:00.541454 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 342/691] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:56:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:56:00.547788 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371626ca-e932-49e4-9259-a6c33b0ac328 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:00.611782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bcae9-68ff-426b-8058-7a516fcf100b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:00.652362 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcd598-7049-49b4-9153-7133eb3f8904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:00.653964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcd598-7049-49b4-9153-7133eb3f8904 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:00.654646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcd598-7049-49b4-9153-7133eb3f8904 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:00.655133 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcd598-7049-49b4-9153-7133eb3f8904 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:00.658272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:00.663926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dcd598-7049-49b4-9153-7133eb3f8904 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:00.665962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dcd598-7049-49b4-9153-7133eb3f8904 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:00.685715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-59dcd598-7049-49b4-9153-7133eb3f8904 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 13:56:00.685715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:00.685715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:00.685715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:00.685715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:00.685715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:00.685715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:00.685715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:00.685715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:00.685715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:00.685715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:00.685715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:00.685715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:56:00.685715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:56:00.685715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:56:00.685715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:56:00.686715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:00.686715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:00.686715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:56:00.686715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:00.686715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:56:00.686715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:00.686715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:00.686715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:00.686715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:00.686715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:00.686715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:00.686715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:00.686715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:00.686715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:00.686715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:00.686715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:00.686715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:00.686715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:00.687771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:00.687771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:00.687771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:56:00.687771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:56:00.687771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:56:00.687771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:56:00.687771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:56:00.687771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:56:00.687771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 13:56:00.687771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:00.690418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 350/692] 10.222.0.94 () {56 vars in 1093 bytes} [Mon Jul 18 13:56:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:00.695918 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcd598-7049-49b4-9153-7133eb3f8904 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:00.756498 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371626ca-e932-49e4-9259-a6c33b0ac328 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:00.793233 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b517e53-7852-47e6-bd38-854cd4687fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:00.794403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b517e53-7852-47e6-bd38-854cd4687fbd None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:00.794897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b517e53-7852-47e6-bd38-854cd4687fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:00.795390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b517e53-7852-47e6-bd38-854cd4687fbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:00.807385 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b517e53-7852-47e6-bd38-854cd4687fbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:00.809372 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b517e53-7852-47e6-bd38-854cd4687fbd None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:00.820258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:00.843566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-7b517e53-7852-47e6-bd38-854cd4687fbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:00.843566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:00.843566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:00.843566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:00.843566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:00.843566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:00.843566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:00.843566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:00.843566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:00.843566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:00.843566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:00.843566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:00.843566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:00.843566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:00.843566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:00.844569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:00.844569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:00.844569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:00.844569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:00.844569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:00.844569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:00.844569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:00.844569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:00.844569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:00.844569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:00.844569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:00.844569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:00.844569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:00.844569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:00.844569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:00.844569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:00.844569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:00.845477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:00.845477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:00.845477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:00.845477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:00.851065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 343/693] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:00.869547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b517e53-7852-47e6-bd38-854cd4687fbd None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:00.937931 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcd598-7049-49b4-9153-7133eb3f8904 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:00.987132 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12618520-3a57-4688-98e2-bda503e05875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:00.987925 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12618520-3a57-4688-98e2-bda503e05875 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:00.988336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12618520-3a57-4688-98e2-bda503e05875 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:00.988628 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12618520-3a57-4688-98e2-bda503e05875 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:00.989474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12618520-3a57-4688-98e2-bda503e05875 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:00.989788 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12618520-3a57-4688-98e2-bda503e05875 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:00.991237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12618520-3a57-4688-98e2-bda503e05875 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:01.009732 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 351/694] 10.222.0.94 () {56 vars in 1133 bytes} [Mon Jul 18 13:56:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:01.011108 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113ae1e-5586-4c41-aae7-ae546cd6f76d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:01.011403 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113ae1e-5586-4c41-aae7-ae546cd6f76d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:01.011789 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113ae1e-5586-4c41-aae7-ae546cd6f76d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:01.013428 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 352/695] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:01.018302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113ae1e-5586-4c41-aae7-ae546cd6f76d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:01.022787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b517e53-7852-47e6-bd38-854cd4687fbd None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:01.063852 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab35b14-4237-4ba7-9cb2-1ccc0208a459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:01.064566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab35b14-4237-4ba7-9cb2-1ccc0208a459 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:01.064858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab35b14-4237-4ba7-9cb2-1ccc0208a459 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:01.065142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab35b14-4237-4ba7-9cb2-1ccc0208a459 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:01.065876 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab35b14-4237-4ba7-9cb2-1ccc0208a459 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:01.066194 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab35b14-4237-4ba7-9cb2-1ccc0208a459 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:01.067591 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab35b14-4237-4ba7-9cb2-1ccc0208a459 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:01.089824 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 344/696] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:01.091290 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4a01a-ba79-43ff-9c98-6324a1a50b63 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:01.091593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4a01a-ba79-43ff-9c98-6324a1a50b63 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:01.091882 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4a01a-ba79-43ff-9c98-6324a1a50b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:01.112713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d9f4a01a-ba79-43ff-9c98-6324a1a50b63 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:01.114837 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d9f4a01a-ba79-43ff-9c98-6324a1a50b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:01.120495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:01.155341 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113ae1e-5586-4c41-aae7-ae546cd6f76d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:01.184144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:01.203807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f12f0-9b86-4ab8-a5fb-37425434c6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:01.204600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f12f0-9b86-4ab8-a5fb-37425434c6ec None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:01.204815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 345/697] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:56:01.204996 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f12f0-9b86-4ab8-a5fb-37425434c6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:01.205283 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f12f0-9b86-4ab8-a5fb-37425434c6ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:01.206287 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4a01a-ba79-43ff-9c98-6324a1a50b63 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:01.284621 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f12f0-9b86-4ab8-a5fb-37425434c6ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:01.286083 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f12f0-9b86-4ab8-a5fb-37425434c6ec None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:01.319236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4a01a-ba79-43ff-9c98-6324a1a50b63 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:01.340028 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:01.359700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405670d2-83a6-4a2e-9f97-bc2a78372037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:01.360846 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405670d2-83a6-4a2e-9f97-bc2a78372037 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:01.361491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405670d2-83a6-4a2e-9f97-bc2a78372037 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:01.361987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405670d2-83a6-4a2e-9f97-bc2a78372037 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:01.369917 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405670d2-83a6-4a2e-9f97-bc2a78372037 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:01.371595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405670d2-83a6-4a2e-9f97-bc2a78372037 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:01.387236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-405670d2-83a6-4a2e-9f97-bc2a78372037 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:01.387236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:01.387236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:01.387236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:01.387236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:01.387236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:01.387236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:01.387236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:01.387236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:01.387236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:01.387236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:01.387236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:01.387236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:56:01.387236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:56:01.387236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:56:01.388195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:56:01.388195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:01.388195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:01.388195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:01.388195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:01.388195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:01.388195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:01.388195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:01.388195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:01.388195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:01.388195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:01.388195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:01.388195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:01.388195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:01.388195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:01.388195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:01.388195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:01.389127 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:56:01.389127 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:56:01.389127 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:56:01.389127 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:56:01.389127 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:56:01.389127 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:56:01.389127 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:01.389127 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:01.390166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 353/698] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:56:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:56:01.391040 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f12f0-9b86-4ab8-a5fb-37425434c6ec None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:01.392020 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 346/699] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:56:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:01.393269 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405670d2-83a6-4a2e-9f97-bc2a78372037 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:01.450079 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:01.491513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f12f0-9b86-4ab8-a5fb-37425434c6ec None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:01.508861 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405670d2-83a6-4a2e-9f97-bc2a78372037 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:01.525760 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6029cfd-3f45-41d2-8bbb-8e07f2f1768b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:01.526558 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6029cfd-3f45-41d2-8bbb-8e07f2f1768b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:01.526843 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6029cfd-3f45-41d2-8bbb-8e07f2f1768b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:01.527119 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6029cfd-3f45-41d2-8bbb-8e07f2f1768b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:01.537828 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6029cfd-3f45-41d2-8bbb-8e07f2f1768b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:01.539317 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6029cfd-3f45-41d2-8bbb-8e07f2f1768b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:01.547477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45720fb-a770-4c01-b90d-2563c892e0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:01.548666 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45720fb-a770-4c01-b90d-2563c892e0fc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:01.549161 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45720fb-a770-4c01-b90d-2563c892e0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:01.549658 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45720fb-a770-4c01-b90d-2563c892e0fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:01.557790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c6029cfd-3f45-41d2-8bbb-8e07f2f1768b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:01.557790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:01.557790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:01.557790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:01.557790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:01.557790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:01.557790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:01.557790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:01.557790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:01.557790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:01.557790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:01.557790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:01.557790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:01.557790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:01.557790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:01.558715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:01.558715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:01.558715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:01.558715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:01.558715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:01.558715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:01.558715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:01.558715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:01.558715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:01.558715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:01.558715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:01.558715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:01.558715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:01.558715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:01.558715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:01.558715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:01.558715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:01.559665 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:01.559665 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:01.559665 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:01.559665 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:01.563742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 354/700] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:01.564565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6029cfd-3f45-41d2-8bbb-8e07f2f1768b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:01.565246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45720fb-a770-4c01-b90d-2563c892e0fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:01.566933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45720fb-a770-4c01-b90d-2563c892e0fc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:01.574712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:01.590308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-e45720fb-a770-4c01-b90d-2563c892e0fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:01.590308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:01.590308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:01.590308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:01.590308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:01.590308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:01.590308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:01.590308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:01.590308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:01.590308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:01.590308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:01.590308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:01.590308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:01.590308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:01.590308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:01.591271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:01.591271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:01.591271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:01.591271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:01.591271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:01.591271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:01.591271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:01.591271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:01.591271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:01.591271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:01.591271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:01.591271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:01.591271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:01.591271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:01.591271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:01.591271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:01.591271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:01.592661 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:01.592661 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:01.592661 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:01.592661 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:01.594775 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 347/701] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:01.595891 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45720fb-a770-4c01-b90d-2563c892e0fc None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:01.674950 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6029cfd-3f45-41d2-8bbb-8e07f2f1768b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:01.709206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45720fb-a770-4c01-b90d-2563c892e0fc None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:01.709577 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c2124-e77f-4d1f-ae06-c96d83d84388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:01.710255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c2124-e77f-4d1f-ae06-c96d83d84388 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:01.710540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c2124-e77f-4d1f-ae06-c96d83d84388 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:01.711348 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c2124-e77f-4d1f-ae06-c96d83d84388 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:01.712162 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c2124-e77f-4d1f-ae06-c96d83d84388 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:01.712475 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c2124-e77f-4d1f-ae06-c96d83d84388 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:01.713060 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:01.713906 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c2124-e77f-4d1f-ae06-c96d83d84388 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:01.727433 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 355/702] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:56:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:01.728386 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c2124-e77f-4d1f-ae06-c96d83d84388 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:01.751970 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4549db-6f4c-4aa7-8a98-94a204b0fdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:01.753161 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4549db-6f4c-4aa7-8a98-94a204b0fdcb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:01.753769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4549db-6f4c-4aa7-8a98-94a204b0fdcb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:01.754307 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4549db-6f4c-4aa7-8a98-94a204b0fdcb None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:01.755287 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4549db-6f4c-4aa7-8a98-94a204b0fdcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:01.755898 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4549db-6f4c-4aa7-8a98-94a204b0fdcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:01.757784 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4549db-6f4c-4aa7-8a98-94a204b0fdcb None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:01.771500 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 348/703] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:01.772730 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4549db-6f4c-4aa7-8a98-94a204b0fdcb None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:01.803680 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:01.840818 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c2124-e77f-4d1f-ae06-c96d83d84388 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:01.888202 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7090a4-1b2b-4e04-81a5-86a162bd40d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:01.889134 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7090a4-1b2b-4e04-81a5-86a162bd40d1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:01.889425 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7090a4-1b2b-4e04-81a5-86a162bd40d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:01.889705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7090a4-1b2b-4e04-81a5-86a162bd40d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:01.890428 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7090a4-1b2b-4e04-81a5-86a162bd40d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:01.890737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7090a4-1b2b-4e04-81a5-86a162bd40d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:01.891917 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4549db-6f4c-4aa7-8a98-94a204b0fdcb None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:01.892200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7090a4-1b2b-4e04-81a5-86a162bd40d1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:01.908034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 356/704] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:01.909184 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7090a4-1b2b-4e04-81a5-86a162bd40d1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:01.937389 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dcd98-3a68-46b0-99ec-dc0be0e332eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:01.938445 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dcd98-3a68-46b0-99ec-dc0be0e332eb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:01.938941 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dcd98-3a68-46b0-99ec-dc0be0e332eb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:01.939425 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dcd98-3a68-46b0-99ec-dc0be0e332eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:01.948919 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0dcd98-3a68-46b0-99ec-dc0be0e332eb None admin] RBAC: Adding query filter params (role.id=5c4ba827ac854bc69cc26c3bcd27dc5c, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=cbf513ee25374e49b3d8ce28843e393a) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:01.949558 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0dcd98-3a68-46b0-99ec-dc0be0e332eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:01.951238 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0dcd98-3a68-46b0-99ec-dc0be0e332eb None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:01.961016 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 349/705] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:56:01 2022] GET /identity/v3/role_assignments?user.id=cbf513ee25374e49b3d8ce28843e393a&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=5c4ba827ac854bc69cc26c3bcd27dc5c => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:01.962564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dcd98-3a68-46b0-99ec-dc0be0e332eb None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:02.003294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:02.042656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7090a4-1b2b-4e04-81a5-86a162bd40d1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:02.078749 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:02.089580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dcd98-3a68-46b0-99ec-dc0be0e332eb None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:02.096474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be409676-d654-449b-9f2e-b67d2770bd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:02.097547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be409676-d654-449b-9f2e-b67d2770bd7e None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:02.098101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be409676-d654-449b-9f2e-b67d2770bd7e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:02.099263 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be409676-d654-449b-9f2e-b67d2770bd7e None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:02.100233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be409676-d654-449b-9f2e-b67d2770bd7e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:02.102439 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be409676-d654-449b-9f2e-b67d2770bd7e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:02.139975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 357/706] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Jul 18 13:56:01 2022] POST /identity/v3/users => generated 254 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:56:02.139975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65257e7c-bdda-40a7-82ec-87f1809b044f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:02.139975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65257e7c-bdda-40a7-82ec-87f1809b044f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:02.139975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65257e7c-bdda-40a7-82ec-87f1809b044f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:02.144215 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5897684-d9ce-463a-83fb-e09bc9df1f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:02.145240 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5897684-d9ce-463a-83fb-e09bc9df1f17 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:02.145470 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 358/707] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:02.145656 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5897684-d9ce-463a-83fb-e09bc9df1f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:02.145934 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5897684-d9ce-463a-83fb-e09bc9df1f17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:02.148583 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374acab-40aa-4c44-bac0-234c32192119 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:02.148904 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374acab-40aa-4c44-bac0-234c32192119 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:02.150852 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374acab-40aa-4c44-bac0-234c32192119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:02.158709 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5897684-d9ce-463a-83fb-e09bc9df1f17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:02.160355 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5897684-d9ce-463a-83fb-e09bc9df1f17 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:02.180247 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d374acab-40aa-4c44-bac0-234c32192119 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:02.181554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d374acab-40aa-4c44-bac0-234c32192119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:02.182204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-b5897684-d9ce-463a-83fb-e09bc9df1f17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:02.182204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:02.182204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:02.182204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:02.182204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:02.182204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:02.182204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:02.182204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:02.182204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:02.182204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:02.182204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:02.182204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:02.182204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:56:02.182204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:56:02.182204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:56:02.183053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:56:02.183053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:02.183053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:02.183053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:02.183053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:02.183053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:02.183053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:02.183053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:02.183053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:02.183053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:02.183053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:02.183053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:02.183053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:02.183053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:02.183053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:02.183053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:02.183053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:02.184595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:56:02.184595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:56:02.184595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:56:02.184595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:56:02.184595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:56:02.184595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:56:02.184595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:02.184595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:02.184595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 350/708] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:56:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:02.190828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5897684-d9ce-463a-83fb-e09bc9df1f17 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:02.270025 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 359/709] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:56:02.271308 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5ef3f-ada3-49fb-8ff1-7c61e727bde9 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:02.271604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5ef3f-ada3-49fb-8ff1-7c61e727bde9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:02.271889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5ef3f-ada3-49fb-8ff1-7c61e727bde9 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:02.274406 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 360/710] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:02.280890 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecee37-90ac-47a7-9a01-6b356fbc93ee None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:02.281185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecee37-90ac-47a7-9a01-6b356fbc93ee None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:02.281474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecee37-90ac-47a7-9a01-6b356fbc93ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:02.304429 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:02.305096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2fecee37-90ac-47a7-9a01-6b356fbc93ee None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:02.306805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2fecee37-90ac-47a7-9a01-6b356fbc93ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:02.325489 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5897684-d9ce-463a-83fb-e09bc9df1f17 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:02.378439 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128f46d-08dd-434a-9bda-3de963159f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:02.379123 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128f46d-08dd-434a-9bda-3de963159f0a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:02.379450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128f46d-08dd-434a-9bda-3de963159f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:02.379984 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128f46d-08dd-434a-9bda-3de963159f0a None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:02.381253 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e128f46d-08dd-434a-9bda-3de963159f0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:02.381853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e128f46d-08dd-434a-9bda-3de963159f0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:02.384047 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e128f46d-08dd-434a-9bda-3de963159f0a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:02.388134 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 361/711] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:56:02.388989 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fecee37-90ac-47a7-9a01-6b356fbc93ee None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:02.402338 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 351/712] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:56:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:02.409578 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128f46d-08dd-434a-9bda-3de963159f0a None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:02.446752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:02.523631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fecee37-90ac-47a7-9a01-6b356fbc93ee None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:02.543929 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128f46d-08dd-434a-9bda-3de963159f0a None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:02.568371 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3752e-a648-498c-baf5-27b5ce1f75d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:02.569278 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3752e-a648-498c-baf5-27b5ce1f75d6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:02.570592 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3752e-a648-498c-baf5-27b5ce1f75d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:02.570951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3752e-a648-498c-baf5-27b5ce1f75d6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:02.571983 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e3752e-a648-498c-baf5-27b5ce1f75d6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:02.573920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e3752e-a648-498c-baf5-27b5ce1f75d6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:02.586298 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aeb9f9-de06-459c-890a-3b80eb964751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:02.586965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aeb9f9-de06-459c-890a-3b80eb964751 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:02.587252 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aeb9f9-de06-459c-890a-3b80eb964751 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:02.587532 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aeb9f9-de06-459c-890a-3b80eb964751 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:02.588410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aeb9f9-de06-459c-890a-3b80eb964751 None admin] RBAC: Adding query filter params (role.id=c60a99de086f45c6820f7872a6b43feb, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=ac717fcd9fc1499db12cf852d45572a8) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:02.588727 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aeb9f9-de06-459c-890a-3b80eb964751 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:02.590181 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aeb9f9-de06-459c-890a-3b80eb964751 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:02.591034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-70e3752e-a648-498c-baf5-27b5ce1f75d6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 13:56:02.591034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:02.591034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:56:02.591034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:56:02.591034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:56:02.591034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:56:02.591034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:56:02.591034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:02.591034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 13:56:02.591034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:02.591034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:02.591034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:02.591034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:02.591034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:02.591034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:02.591034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:02.593182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:02.593182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:02.593182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:02.593182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:02.593182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:02.593182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:02.593182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:02.593182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:56:02.593182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:56:02.593182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:56:02.593182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:56:02.593182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:02.593182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:02.593182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:02.593182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:02.593182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:02.595182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:02.595182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:02.595182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:02.595182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:02.595182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:02.595182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:02.595182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:02.595182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:02.595182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:02.595182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:02.595182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:02.595182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:56:02.595182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:02.595182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 13:56:02.595182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:02.595182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 362/713] 10.222.0.94 () {56 vars in 1129 bytes} [Mon Jul 18 13:56:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:02.597279 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3752e-a648-498c-baf5-27b5ce1f75d6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:02.597373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 352/714] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:56:02 2022] GET /identity/v3/role_assignments?user.id=ac717fcd9fc1499db12cf852d45572a8&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=c60a99de086f45c6820f7872a6b43feb => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:02.603874 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aeb9f9-de06-459c-890a-3b80eb964751 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:02.635114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:02.720755 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3752e-a648-498c-baf5-27b5ce1f75d6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:02.729712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:02.761883 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b467d-073a-46b7-90b3-ed05dbe8cd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:02.762813 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b467d-073a-46b7-90b3-ed05dbe8cd82 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:02.763159 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b467d-073a-46b7-90b3-ed05dbe8cd82 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:02.763512 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b467d-073a-46b7-90b3-ed05dbe8cd82 None admin] PATH_INFO: `/v3/services/network` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:02.764301 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b467d-073a-46b7-90b3-ed05dbe8cd82 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:02.765834 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b467d-073a-46b7-90b3-ed05dbe8cd82 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:02.778742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-0a0b467d-073a-46b7-90b3-ed05dbe8cd82 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 13:56:02.778742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:02.778742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:56:02.778742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:56:02.778742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:56:02.778742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:56:02.778742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:56:02.778742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:02.778742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 13:56:02.778742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:02.778742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:02.778742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:02.778742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:02.778742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:02.778742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:02.778742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:02.778742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:02.779791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:02.779791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:02.779791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:02.779791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:02.779791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:02.779791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:02.779791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:56:02.779791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:56:02.779791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:56:02.779791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:56:02.779791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:02.779791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:02.779791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:02.779791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:02.779791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:02.779791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:02.780686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:02.780686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:02.780686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:02.780686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:02.780686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:02.780686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:02.780686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:02.780686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:02.780686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:02.780686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:02.780686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:56:02.780686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:02.780686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 13:56:02.780686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:02.780686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 363/715] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:56:02 2022] GET /identity/v3/services/network => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:02.786567 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b467d-073a-46b7-90b3-ed05dbe8cd82 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:02.857894 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aeb9f9-de06-459c-890a-3b80eb964751 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:02.901121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c2966-f2d4-4a90-931b-50672da0653b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:02.901793 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c2966-f2d4-4a90-931b-50672da0653b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:02.902081 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c2966-f2d4-4a90-931b-50672da0653b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:02.902382 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c2966-f2d4-4a90-931b-50672da0653b None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:02.903097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c2966-f2d4-4a90-931b-50672da0653b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:02.903410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c2966-f2d4-4a90-931b-50672da0653b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:02.904817 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c2966-f2d4-4a90-931b-50672da0653b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:02.921051 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 353/716] 10.222.0.94 () {56 vars in 1124 bytes} [Mon Jul 18 13:56:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:02.923987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c2966-f2d4-4a90-931b-50672da0653b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:03.014639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:03.045664 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c2966-f2d4-4a90-931b-50672da0653b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:03.059365 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b467d-073a-46b7-90b3-ed05dbe8cd82 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:03.083281 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e43b1-e69e-41fc-acbc-0c19b262c0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:03.084305 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e43b1-e69e-41fc-acbc-0c19b262c0e8 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:03.084767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e43b1-e69e-41fc-acbc-0c19b262c0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:03.085204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e43b1-e69e-41fc-acbc-0c19b262c0e8 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:03.086585 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e43b1-e69e-41fc-acbc-0c19b262c0e8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:03.087090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e43b1-e69e-41fc-acbc-0c19b262c0e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:03.088655 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e43b1-e69e-41fc-acbc-0c19b262c0e8 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:03.095252 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 354/717] 10.222.0.94 () {56 vars in 1124 bytes} [Mon Jul 18 13:56:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:03.117411 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc63b76-bebc-4400-8611-f21a55b9b9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:03.118091 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc63b76-bebc-4400-8611-f21a55b9b9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:03.118377 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc63b76-bebc-4400-8611-f21a55b9b9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:03.118651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc63b76-bebc-4400-8611-f21a55b9b9a1 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:03.119365 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc63b76-bebc-4400-8611-f21a55b9b9a1 None admin] RBAC: Adding query filter params (name=network) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:03.119676 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc63b76-bebc-4400-8611-f21a55b9b9a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:03.121144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc63b76-bebc-4400-8611-f21a55b9b9a1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:03.130051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 364/718] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Jul 18 13:56:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:03.132209 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e43b1-e69e-41fc-acbc-0c19b262c0e8 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:03.243962 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e43b1-e69e-41fc-acbc-0c19b262c0e8 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:03.282232 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04fe12-571d-4978-a9c9-b406460b1a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:03.283175 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04fe12-571d-4978-a9c9-b406460b1a40 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:03.283611 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04fe12-571d-4978-a9c9-b406460b1a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:03.284068 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04fe12-571d-4978-a9c9-b406460b1a40 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:03.284965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04fe12-571d-4978-a9c9-b406460b1a40 None admin] RBAC: Adding query filter params (type=network) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:03.285425 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04fe12-571d-4978-a9c9-b406460b1a40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:03.286945 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04fe12-571d-4978-a9c9-b406460b1a40 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:03.293222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 355/719] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Jul 18 13:56:03 2022] GET /identity/v3/services?type=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:04.393999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6cd42-7bae-4ad9-b24c-7fc3a938b196 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:04.394532 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6cd42-7bae-4ad9-b24c-7fc3a938b196 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:04.394614 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6cd42-7bae-4ad9-b24c-7fc3a938b196 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:04.396230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 365/720] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:04.401667 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a0853-e199-4c66-9e17-e8537fe068af None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:04.402065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a0853-e199-4c66-9e17-e8537fe068af None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:04.402251 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a0853-e199-4c66-9e17-e8537fe068af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:04.417442 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:04.432901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-8f3a0853-e199-4c66-9e17-e8537fe068af None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:04.434119 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8f3a0853-e199-4c66-9e17-e8537fe068af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:04.470722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73a63b-ac76-49cb-9338-226f25bb8ff5 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:04.471067 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73a63b-ac76-49cb-9338-226f25bb8ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:04.471288 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73a63b-ac76-49cb-9338-226f25bb8ff5 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:04.473273 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 366/721] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:56:04.480156 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8ce87-11c8-4d5b-8e60-ef50eed0e7bc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:04.480451 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8ce87-11c8-4d5b-8e60-ef50eed0e7bc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:04.480735 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8ce87-11c8-4d5b-8e60-ef50eed0e7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:04.515600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-52a8ce87-11c8-4d5b-8e60-ef50eed0e7bc None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:04.516834 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-52a8ce87-11c8-4d5b-8e60-ef50eed0e7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:04.535129 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 356/722] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 13:56:04.559447 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a0853-e199-4c66-9e17-e8537fe068af None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:04.586538 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:04.669283 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 367/723] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 13:56:04.719745 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a0853-e199-4c66-9e17-e8537fe068af None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:04.726797 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8ce87-11c8-4d5b-8e60-ef50eed0e7bc None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:04.784302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:04.794900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ea771-cd07-423b-89fb-a5aad3a48a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:04.795720 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ea771-cd07-423b-89fb-a5aad3a48a1b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:04.796041 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ea771-cd07-423b-89fb-a5aad3a48a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:04.796732 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ea771-cd07-423b-89fb-a5aad3a48a1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:04.816074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ea771-cd07-423b-89fb-a5aad3a48a1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:56:04.832465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-7d8ea771-cd07-423b-89fb-a5aad3a48a1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:56:04.832465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:04.832465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:04.832465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:04.832465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:04.832465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:04.832465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:04.832465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:04.832465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:04.832465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:04.832465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:04.832465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:04.832465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:56:04.832465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:56:04.832465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:56:04.832465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:56:04.833419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:56:04.833419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:56:04.833419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:56:04.833419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:56:04.833419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:04.833419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:04.833419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:04.833419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:04.833419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:04.833419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:04.833419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:04.833419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:04.833419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:04.833419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:04.833419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:04.833419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:04.834259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:04.834259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:04.834259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:04.834259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:04.834259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:56:04.834259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:56:04.834259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:56:04.834259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:56:04.834259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:56:04.834259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:56:04.834259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:56:04.834259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:04.834915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 357/724] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:56:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:04.843129 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ea771-cd07-423b-89fb-a5aad3a48a1b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:04.905778 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8ce87-11c8-4d5b-8e60-ef50eed0e7bc None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:04.980788 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0291c-01c7-4193-8f8a-1e796f470b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:04.981514 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0291c-01c7-4193-8f8a-1e796f470b60 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:04.981598 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0291c-01c7-4193-8f8a-1e796f470b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:04.981846 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0291c-01c7-4193-8f8a-1e796f470b60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:05.006812 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0291c-01c7-4193-8f8a-1e796f470b60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:56:05.014927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e7a0291c-01c7-4193-8f8a-1e796f470b60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:56:05.014927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:05.014927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:05.014927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:05.014927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:05.014927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:05.014927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:05.014927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:05.014927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:05.014927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:05.014927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:05.014927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:05.014927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:56:05.014927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:56:05.014927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:56:05.014927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:56:05.015881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:56:05.015881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:56:05.015881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:56:05.015881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:56:05.015881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:05.015881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:05.015881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:05.015881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:05.015881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:05.015881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:05.015881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:05.015881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:05.015881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:05.015881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:05.015881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:05.015881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:05.016874 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:05.016874 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:05.016874 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:05.016874 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:05.016874 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:56:05.016874 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:56:05.016874 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:56:05.016874 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:56:05.016874 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:56:05.016874 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:56:05.016874 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:56:05.016874 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:05.028973 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 368/725] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:56:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:05.034506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0291c-01c7-4193-8f8a-1e796f470b60 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:05.052847 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ea771-cd07-423b-89fb-a5aad3a48a1b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:05.088927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:05.113384 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4a547-0e47-4ba1-b693-43e6d11da4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:05.114332 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4a547-0e47-4ba1-b693-43e6d11da4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:05.114619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4a547-0e47-4ba1-b693-43e6d11da4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:05.114959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4a547-0e47-4ba1-b693-43e6d11da4e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:05.115880 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4a547-0e47-4ba1-b693-43e6d11da4e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:05.116605 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4a547-0e47-4ba1-b693-43e6d11da4e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:05.118831 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4a547-0e47-4ba1-b693-43e6d11da4e8 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:05.129740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 358/726] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Jul 18 13:56:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:05.131103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952bc80-21a2-473d-a989-1e5508e1bad9 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:05.131397 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952bc80-21a2-473d-a989-1e5508e1bad9 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:05.131677 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952bc80-21a2-473d-a989-1e5508e1bad9 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:05.133796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 359/727] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:05.135367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952bc80-21a2-473d-a989-1e5508e1bad9 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:05.183479 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0291c-01c7-4193-8f8a-1e796f470b60 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:05.204401 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:05.223357 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00efdc6-3364-4e99-9eb4-2417a518a086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:05.224065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00efdc6-3364-4e99-9eb4-2417a518a086 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:05.224357 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00efdc6-3364-4e99-9eb4-2417a518a086 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:05.224640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00efdc6-3364-4e99-9eb4-2417a518a086 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:05.225392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00efdc6-3364-4e99-9eb4-2417a518a086 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:05.225705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00efdc6-3364-4e99-9eb4-2417a518a086 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:05.227264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00efdc6-3364-4e99-9eb4-2417a518a086 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:05.240861 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 369/728] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:56:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:05.242713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a379867-0726-42b8-a540-6eebe56603b1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:05.243011 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a379867-0726-42b8-a540-6eebe56603b1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:05.243396 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a379867-0726-42b8-a540-6eebe56603b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:05.258051 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952bc80-21a2-473d-a989-1e5508e1bad9 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:05.260905 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7a379867-0726-42b8-a540-6eebe56603b1 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:05.262161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7a379867-0726-42b8-a540-6eebe56603b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:05.298343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5259b-9797-4960-8b1c-c5649fce24ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:05.299295 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5259b-9797-4960-8b1c-c5649fce24ed None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:05.299579 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5259b-9797-4960-8b1c-c5649fce24ed None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:05.299859 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5259b-9797-4960-8b1c-c5649fce24ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:05.313598 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5259b-9797-4960-8b1c-c5649fce24ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:05.315050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5259b-9797-4960-8b1c-c5649fce24ed None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:05.333343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-c0f5259b-9797-4960-8b1c-c5649fce24ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:05.333343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:05.333343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:05.333343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:05.333343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:05.333343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:05.333343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:05.333343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:05.333343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:05.333343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:05.333343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:05.333343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:05.333343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:05.333343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:05.333343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:05.335374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:05.335374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:05.335374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:05.335374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:05.335374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:05.335374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:05.335374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:05.335374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:05.335374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:05.335374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:05.335374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:05.335374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:05.335374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:05.335374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:05.335374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:05.335374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:05.335374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:05.337553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:05.337553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:05.337553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:05.337553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:05.344004 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 370/730] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:56:05.344004 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a379867-0726-42b8-a540-6eebe56603b1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:05.344370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 360/730] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:05.345062 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5259b-9797-4960-8b1c-c5649fce24ed None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:05.358604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:05.470232 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5259b-9797-4960-8b1c-c5649fce24ed None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:05.470873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a379867-0726-42b8-a540-6eebe56603b1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:05.478160 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:05.509031 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa4851-3f90-445f-a5b4-5040ff64dcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:05.509880 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa4851-3f90-445f-a5b4-5040ff64dcb6 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:05.510285 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa4851-3f90-445f-a5b4-5040ff64dcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:05.510637 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa4851-3f90-445f-a5b4-5040ff64dcb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:05.511443 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa4851-3f90-445f-a5b4-5040ff64dcb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:05.511823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa4851-3f90-445f-a5b4-5040ff64dcb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:05.513330 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa4851-3f90-445f-a5b4-5040ff64dcb6 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:05.516647 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf0695-aff6-4a81-82c7-bcd1d001efde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:05.517449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf0695-aff6-4a81-82c7-bcd1d001efde None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:05.517734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf0695-aff6-4a81-82c7-bcd1d001efde None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:05.518010 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf0695-aff6-4a81-82c7-bcd1d001efde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:05.526029 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 361/731] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:05.526721 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa4851-3f90-445f-a5b4-5040ff64dcb6 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:05.530325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf0695-aff6-4a81-82c7-bcd1d001efde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:05.531777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf0695-aff6-4a81-82c7-bcd1d001efde None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:05.550161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b7bf0695-aff6-4a81-82c7-bcd1d001efde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:05.550161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:05.550161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:05.550161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:05.550161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:05.550161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:05.550161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:05.550161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:05.550161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:05.550161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:05.550161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:05.550161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:05.550161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:05.550161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:05.550161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:05.552501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:05.552501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:05.552501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:05.552501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:05.552501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:05.552501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:05.552501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:05.552501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:05.552501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:05.552501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:05.552501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:05.552501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:05.552501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:05.552501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:05.552501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:05.552501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:05.552501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:05.554761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:05.554761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:05.554761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:05.554761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:05.554761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 371/732] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:05.554761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5801c-e2ed-4a11-b9d1-397a70155842 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:05.554761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5801c-e2ed-4a11-b9d1-397a70155842 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:05.554761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5801c-e2ed-4a11-b9d1-397a70155842 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:05.554761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 372/733] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:05.561105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5801c-e2ed-4a11-b9d1-397a70155842 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:05.635656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:05.643237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa4851-3f90-445f-a5b4-5040ff64dcb6 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:05.673534 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5801c-e2ed-4a11-b9d1-397a70155842 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:05.678227 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d22d05-ed07-4823-9890-72a0ca0be45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:05.679021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d22d05-ed07-4823-9890-72a0ca0be45e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:05.679304 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d22d05-ed07-4823-9890-72a0ca0be45e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:05.679578 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d22d05-ed07-4823-9890-72a0ca0be45e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:05.689500 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d22d05-ed07-4823-9890-72a0ca0be45e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:56:05.697516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-f5d22d05-ed07-4823-9890-72a0ca0be45e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:56:05.697516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:05.697516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:05.697516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:05.697516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:05.697516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:05.697516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:05.697516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:05.697516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:05.697516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:05.697516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:05.697516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:05.697516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:56:05.697516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:56:05.697516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:56:05.697516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:56:05.698428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:56:05.698428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:56:05.698428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:56:05.698428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:56:05.698428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:05.698428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:05.698428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:05.698428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:05.698428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:05.698428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:05.698428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:05.698428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:05.698428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:05.698428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:05.698428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:05.698428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:05.699263 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:05.699263 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:05.699263 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:05.699263 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:05.699263 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:56:05.699263 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:56:05.699263 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:56:05.699263 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:56:05.699263 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:56:05.699263 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:56:05.699263 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:56:05.699263 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:05.699263 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 362/734] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:56:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:05.700412 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d22d05-ed07-4823-9890-72a0ca0be45e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:05.716093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9ebcf-f2c3-4b89-9bec-e2831d9c3d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:05.717419 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9ebcf-f2c3-4b89-9bec-e2831d9c3d04 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:05.717710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9ebcf-f2c3-4b89-9bec-e2831d9c3d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:05.717988 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9ebcf-f2c3-4b89-9bec-e2831d9c3d04 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:05.725601 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9ebcf-f2c3-4b89-9bec-e2831d9c3d04 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:05.727306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9ebcf-f2c3-4b89-9bec-e2831d9c3d04 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:05.728982 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:05.743463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-29d9ebcf-f2c3-4b89-9bec-e2831d9c3d04 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 13:56:05.743463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:05.743463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:05.743463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:05.743463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:05.743463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:05.743463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:05.743463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:05.743463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:05.743463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:05.743463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:05.743463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:05.743463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:56:05.743463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:56:05.743463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:56:05.743463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:56:05.744495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:05.744495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:05.744495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:56:05.744495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:05.744495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:56:05.744495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:05.744495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:05.744495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:05.744495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:05.744495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:05.744495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:05.744495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:05.744495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:05.744495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:05.744495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:05.744495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:05.744495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:05.744495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:05.745530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:05.745530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:05.745530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:56:05.745530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:56:05.745530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:56:05.745530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:56:05.745530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:56:05.745530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:56:05.745530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 13:56:05.745530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:05.749336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 373/735] 10.222.0.94 () {56 vars in 1117 bytes} [Mon Jul 18 13:56:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:05.750403 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c0735-a653-40e5-9a34-de37ddd66b84 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:05.750684 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c0735-a653-40e5-9a34-de37ddd66b84 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:05.750968 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c0735-a653-40e5-9a34-de37ddd66b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:05.767339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-679c0735-a653-40e5-9a34-de37ddd66b84 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:05.768532 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-679c0735-a653-40e5-9a34-de37ddd66b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:05.818417 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d22d05-ed07-4823-9890-72a0ca0be45e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:05.835054 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 374/736] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:56:05.836465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c0735-a653-40e5-9a34-de37ddd66b84 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:05.871690 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d921816-c3f2-4db8-9b38-a8dcc1c4eed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:05.872822 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d921816-c3f2-4db8-9b38-a8dcc1c4eed1 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:05.873261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d921816-c3f2-4db8-9b38-a8dcc1c4eed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:05.873715 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d921816-c3f2-4db8-9b38-a8dcc1c4eed1 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:05.874586 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d921816-c3f2-4db8-9b38-a8dcc1c4eed1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:05.875058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d921816-c3f2-4db8-9b38-a8dcc1c4eed1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:05.876543 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d921816-c3f2-4db8-9b38-a8dcc1c4eed1 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:05.891299 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 363/737] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:05.892466 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d921816-c3f2-4db8-9b38-a8dcc1c4eed1 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:05.909040 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:05.960039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c0735-a653-40e5-9a34-de37ddd66b84 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:05.987896 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:05.994985 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede391fe-553f-426b-9ede-c7e175799c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:05.995660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede391fe-553f-426b-9ede-c7e175799c93 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:05.995967 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede391fe-553f-426b-9ede-c7e175799c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:05.996267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede391fe-553f-426b-9ede-c7e175799c93 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:05.997014 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede391fe-553f-426b-9ede-c7e175799c93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:05.997327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede391fe-553f-426b-9ede-c7e175799c93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:05.998700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede391fe-553f-426b-9ede-c7e175799c93 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:06.002823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d921816-c3f2-4db8-9b38-a8dcc1c4eed1 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:06.006742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 375/738] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:56:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:06.007541 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede391fe-553f-426b-9ede-c7e175799c93 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:06.036952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c631d95-812a-4d56-98e2-7daaa9b4c3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:06.037633 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c631d95-812a-4d56-98e2-7daaa9b4c3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:06.037916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c631d95-812a-4d56-98e2-7daaa9b4c3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:06.038199 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c631d95-812a-4d56-98e2-7daaa9b4c3d5 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:06.039619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c631d95-812a-4d56-98e2-7daaa9b4c3d5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:06.039930 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c631d95-812a-4d56-98e2-7daaa9b4c3d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:06.041388 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c631d95-812a-4d56-98e2-7daaa9b4c3d5 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:06.058660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 364/739] 10.222.0.94 () {56 vars in 1149 bytes} [Mon Jul 18 13:56:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:06.059457 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c631d95-812a-4d56-98e2-7daaa9b4c3d5 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:06.105219 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede391fe-553f-426b-9ede-c7e175799c93 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:06.124140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:06.139055 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e25d4-858c-4124-9158-9573dab4edfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:06.139722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e25d4-858c-4124-9158-9573dab4edfd None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:06.140039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e25d4-858c-4124-9158-9573dab4edfd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:06.140324 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e25d4-858c-4124-9158-9573dab4edfd None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:06.141053 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e25d4-858c-4124-9158-9573dab4edfd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:06.142423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e25d4-858c-4124-9158-9573dab4edfd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:06.157201 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 376/740] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Jul 18 13:56:06 2022] POST /identity/v3/services => generated 251 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:56:06.158690 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117257e-0381-4fb5-9c6d-2330f24b6040 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:06.158994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117257e-0381-4fb5-9c6d-2330f24b6040 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:06.159278 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117257e-0381-4fb5-9c6d-2330f24b6040 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:06.160574 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 377/741] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:06.161302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117257e-0381-4fb5-9c6d-2330f24b6040 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:06.169568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c631d95-812a-4d56-98e2-7daaa9b4c3d5 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:06.212716 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3f9d8-0bc3-48e1-be30-1fda160f30e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:06.213544 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3f9d8-0bc3-48e1-be30-1fda160f30e5 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:06.213830 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3f9d8-0bc3-48e1-be30-1fda160f30e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:06.214106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3f9d8-0bc3-48e1-be30-1fda160f30e5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:06.222277 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3f9d8-0bc3-48e1-be30-1fda160f30e5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:06.223754 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3f9d8-0bc3-48e1-be30-1fda160f30e5 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:06.229394 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:06.244943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-80a3f9d8-0bc3-48e1-be30-1fda160f30e5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 13:56:06.244943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:06.244943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:06.244943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:06.244943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:06.244943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:06.244943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:06.244943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:06.244943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:06.244943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:06.244943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:06.244943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:06.244943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:56:06.244943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:56:06.244943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:56:06.244943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:56:06.245943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:06.245943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:06.245943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:56:06.245943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:06.245943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:56:06.245943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:06.245943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:06.245943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:06.245943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:06.245943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:06.245943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:06.245943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:06.245943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:06.245943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:06.245943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:06.245943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:06.245943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:06.245943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:06.246913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:06.246913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:06.246913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:56:06.246913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:56:06.246913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:56:06.246913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:56:06.246913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:56:06.246913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:56:06.246913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 13:56:06.246913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:06.246913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 365/742] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:56:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:06.246913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3f9d8-0bc3-48e1-be30-1fda160f30e5 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:06.286698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117257e-0381-4fb5-9c6d-2330f24b6040 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:06.327706 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a66ee-f363-4d9e-9188-64ff0b2bee8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:06.328930 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a66ee-f363-4d9e-9188-64ff0b2bee8c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:06.329713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a66ee-f363-4d9e-9188-64ff0b2bee8c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:06.329997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a66ee-f363-4d9e-9188-64ff0b2bee8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:06.345077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a66ee-f363-4d9e-9188-64ff0b2bee8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:06.347094 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a66ee-f363-4d9e-9188-64ff0b2bee8c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:06.360801 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3f9d8-0bc3-48e1-be30-1fda160f30e5 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:06.365623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e00a66ee-f363-4d9e-9188-64ff0b2bee8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:06.365623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:06.365623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:06.365623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:06.365623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:06.365623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:06.365623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:06.365623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:06.365623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:06.365623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:06.365623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:06.365623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:06.365623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:06.365623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:06.365623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:06.366500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:06.366500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:06.366500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:06.366500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:06.366500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:06.366500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:06.366500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:06.366500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:06.366500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:06.366500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:06.366500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:06.366500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:06.366500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:06.366500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:06.366500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:06.366500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:06.366500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:06.372880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:06.372880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:06.372880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:06.372880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:06.372880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 378/743] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:06.372880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb558c-2e5d-4929-960a-102359db5df1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:06.372880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb558c-2e5d-4929-960a-102359db5df1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:06.372880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb558c-2e5d-4929-960a-102359db5df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:06.388643 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-36fb558c-2e5d-4929-960a-102359db5df1 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:06.389855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-36fb558c-2e5d-4929-960a-102359db5df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:06.404278 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b81dfb-887f-4199-a250-a0d501549bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:06.405689 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b81dfb-887f-4199-a250-a0d501549bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:06.406434 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b81dfb-887f-4199-a250-a0d501549bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:06.406912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b81dfb-887f-4199-a250-a0d501549bf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:06.416943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:06.468002 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 379/744] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 13:56:06.468002 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb558c-2e5d-4929-960a-102359db5df1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:06.485463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:06.488364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b81dfb-887f-4199-a250-a0d501549bf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:06.490074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b81dfb-887f-4199-a250-a0d501549bf2 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:06.586913 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb558c-2e5d-4929-960a-102359db5df1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:06.598867 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 366/745] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:56:06 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 13:56:06.604323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b81dfb-887f-4199-a250-a0d501549bf2 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:06.623078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59b6ac-0da4-4f6e-9d95-8f6f772c5c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:06.624022 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59b6ac-0da4-4f6e-9d95-8f6f772c5c36 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:06.624201 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59b6ac-0da4-4f6e-9d95-8f6f772c5c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:06.624461 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59b6ac-0da4-4f6e-9d95-8f6f772c5c36 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:06.625309 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db59b6ac-0da4-4f6e-9d95-8f6f772c5c36 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:06.625626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db59b6ac-0da4-4f6e-9d95-8f6f772c5c36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:06.627986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db59b6ac-0da4-4f6e-9d95-8f6f772c5c36 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:06.647066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 380/746] 10.222.0.94 () {56 vars in 1135 bytes} [Mon Jul 18 13:56:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:06.648631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59b6ac-0da4-4f6e-9d95-8f6f772c5c36 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:06.672823 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:06.712447 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:06.716630 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b81dfb-887f-4199-a250-a0d501549bf2 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:06.754798 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59b6ac-0da4-4f6e-9d95-8f6f772c5c36 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:06.755013 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45cd7ad-2b9f-4705-b9bf-b46256e40b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:06.755750 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cd7ad-2b9f-4705-b9bf-b46256e40b6e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:06.756243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cd7ad-2b9f-4705-b9bf-b46256e40b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:06.756431 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cd7ad-2b9f-4705-b9bf-b46256e40b6e None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:06.757147 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45cd7ad-2b9f-4705-b9bf-b46256e40b6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:06.758539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45cd7ad-2b9f-4705-b9bf-b46256e40b6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:06.759432 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45cd7ad-2b9f-4705-b9bf-b46256e40b6e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:06.775441 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 367/747] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:06.776162 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45cd7ad-2b9f-4705-b9bf-b46256e40b6e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:06.791591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8dbb5-ece1-40e4-a00a-8ac38d3f870b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:06.792299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8dbb5-ece1-40e4-a00a-8ac38d3f870b None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:06.792588 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8dbb5-ece1-40e4-a00a-8ac38d3f870b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:06.792873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b8dbb5-ece1-40e4-a00a-8ac38d3f870b None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:06.793595 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b8dbb5-ece1-40e4-a00a-8ac38d3f870b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:06.794967 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b8dbb5-ece1-40e4-a00a-8ac38d3f870b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:06.809495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 381/748] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Jul 18 13:56:06 2022] POST /identity/v3/services => generated 240 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:56:06.810304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8dbb5-ece1-40e4-a00a-8ac38d3f870b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:06.890245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45cd7ad-2b9f-4705-b9bf-b46256e40b6e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:06.907858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:06.935232 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565c4cf-028a-4b0f-893f-6d6bb80d2546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:06.936098 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565c4cf-028a-4b0f-893f-6d6bb80d2546 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:06.936385 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565c4cf-028a-4b0f-893f-6d6bb80d2546 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:06.936662 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565c4cf-028a-4b0f-893f-6d6bb80d2546 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:06.950535 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b8dbb5-ece1-40e4-a00a-8ac38d3f870b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:06.955727 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565c4cf-028a-4b0f-893f-6d6bb80d2546 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:06.957294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565c4cf-028a-4b0f-893f-6d6bb80d2546 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:06.976258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-d565c4cf-028a-4b0f-893f-6d6bb80d2546 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:06.976258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:06.976258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:06.976258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:06.976258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:06.976258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:06.976258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:06.976258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:06.976258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:06.976258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:06.976258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:06.976258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:06.976258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:06.976258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:06.976258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:06.977423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:06.977423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:06.977423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:06.977423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:06.977423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:06.977423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:06.977423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:06.977423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:06.977423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:06.977423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:06.977423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:06.977423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:06.977423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:06.977423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:06.977423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:06.977423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:06.977423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:06.978552 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:06.978552 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:06.978552 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:06.978552 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:06.978552 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 368/749] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:06.978985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565c4cf-028a-4b0f-893f-6d6bb80d2546 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:06.991155 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:06.992314 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9a4da-ec4c-48fa-aeea-4ed76716af36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:06.993167 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9a4da-ec4c-48fa-aeea-4ed76716af36 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:06.993468 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9a4da-ec4c-48fa-aeea-4ed76716af36 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:06.993809 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9a4da-ec4c-48fa-aeea-4ed76716af36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:07.076035 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9a4da-ec4c-48fa-aeea-4ed76716af36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:07.077868 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9a4da-ec4c-48fa-aeea-4ed76716af36 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:07.111885 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565c4cf-028a-4b0f-893f-6d6bb80d2546 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:07.152884 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdaa7d6-7490-4598-bff6-0d0de32e2416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:07.154041 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdaa7d6-7490-4598-bff6-0d0de32e2416 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:07.154535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdaa7d6-7490-4598-bff6-0d0de32e2416 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:07.155014 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdaa7d6-7490-4598-bff6-0d0de32e2416 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:07.163236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdaa7d6-7490-4598-bff6-0d0de32e2416 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:07.164248 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:07.165241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdaa7d6-7490-4598-bff6-0d0de32e2416 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:07.180025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-6fdaa7d6-7490-4598-bff6-0d0de32e2416 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 13:56:07.180025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:07.180025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:07.180025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:07.180025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:07.180025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:07.180025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:07.180025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:07.180025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:07.180025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:07.180025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:07.180025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:07.180025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:56:07.180025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:56:07.180025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:56:07.180025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:56:07.181036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:07.181036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:07.181036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:56:07.181036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:07.181036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:56:07.181036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:07.181036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:07.181036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:07.181036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:07.181036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:07.181036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:07.181036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:07.181036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:07.181036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:07.181036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:07.181036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:07.181036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:07.181036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:07.181987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:07.181987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:07.181987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:56:07.181987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:56:07.181987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:56:07.181987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:56:07.181987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:56:07.181987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:56:07.181987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 13:56:07.181987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:07.181987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 369/750] 10.222.0.94 () {56 vars in 1093 bytes} [Mon Jul 18 13:56:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:07.182575 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdaa7d6-7490-4598-bff6-0d0de32e2416 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:07.192230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 382/751] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:56:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 13:56:07.193362 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9a4da-ec4c-48fa-aeea-4ed76716af36 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:07.253268 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:07.295114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdaa7d6-7490-4598-bff6-0d0de32e2416 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:07.296943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9a4da-ec4c-48fa-aeea-4ed76716af36 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:07.334293 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6c0fb-ae6c-4a10-9900-d9b4acad5ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:07.335073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6c0fb-ae6c-4a10-9900-d9b4acad5ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:07.335388 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6c0fb-ae6c-4a10-9900-d9b4acad5ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:07.335757 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6c0fb-ae6c-4a10-9900-d9b4acad5ec3 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:07.336647 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6c0fb-ae6c-4a10-9900-d9b4acad5ec3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:07.336964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6c0fb-ae6c-4a10-9900-d9b4acad5ec3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:07.337456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df9073-8fae-4623-adc0-8c8ef6a5a6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:07.338320 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df9073-8fae-4623-adc0-8c8ef6a5a6fb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:07.338434 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6c0fb-ae6c-4a10-9900-d9b4acad5ec3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:07.338897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df9073-8fae-4623-adc0-8c8ef6a5a6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:07.339346 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df9073-8fae-4623-adc0-8c8ef6a5a6fb None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:07.340526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df9073-8fae-4623-adc0-8c8ef6a5a6fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:07.341127 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df9073-8fae-4623-adc0-8c8ef6a5a6fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:07.342732 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df9073-8fae-4623-adc0-8c8ef6a5a6fb None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:07.356771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 370/752] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:07.357924 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df9073-8fae-4623-adc0-8c8ef6a5a6fb None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:07.362924 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 383/753] 10.222.0.94 () {56 vars in 1133 bytes} [Mon Jul 18 13:56:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:07.368978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6c0fb-ae6c-4a10-9900-d9b4acad5ec3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:07.403895 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:07.471800 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df9073-8fae-4623-adc0-8c8ef6a5a6fb None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:07.472020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6c0fb-ae6c-4a10-9900-d9b4acad5ec3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:07.501469 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:07.508159 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ce944-4551-41e8-96ed-8ea76e0439a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:07.509065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ce944-4551-41e8-96ed-8ea76e0439a2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:07.509354 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ce944-4551-41e8-96ed-8ea76e0439a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:07.509634 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ce944-4551-41e8-96ed-8ea76e0439a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:07.512485 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab3274-26a6-4708-ba86-7679f3f9ddb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:07.513445 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab3274-26a6-4708-ba86-7679f3f9ddb1 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:07.513873 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab3274-26a6-4708-ba86-7679f3f9ddb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:07.514290 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab3274-26a6-4708-ba86-7679f3f9ddb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:07.521038 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ce944-4551-41e8-96ed-8ea76e0439a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:07.523432 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ce944-4551-41e8-96ed-8ea76e0439a2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:07.526556 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ab3274-26a6-4708-ba86-7679f3f9ddb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:07.528234 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ab3274-26a6-4708-ba86-7679f3f9ddb1 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:07.543646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e38ce944-4551-41e8-96ed-8ea76e0439a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:07.543646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:07.543646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:07.543646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:07.543646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:07.543646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:07.543646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:07.543646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:07.543646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:07.543646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:07.543646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:07.543646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:07.543646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:07.543646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:07.543646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:07.544596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:07.544596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:07.544596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:07.544596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:07.544596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:07.544596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:07.544596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:07.544596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:07.544596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:07.544596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:07.544596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:07.544596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:07.544596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:07.544596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:07.544596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:07.544596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:07.544596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:07.545481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:07.545481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:07.545481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:07.545481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:07.549048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-99ab3274-26a6-4708-ba86-7679f3f9ddb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:07.549048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:07.549048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:07.549048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:07.549048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:07.549048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:07.549048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:07.549048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:07.549048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:07.549048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:07.549048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:07.549048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:07.549048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:07.549048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:07.549048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:07.549913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:07.549913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:07.549913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:07.549913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:07.549913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:07.549913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:07.549913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:07.549913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:07.549913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:07.549913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:07.549913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:07.549913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:07.549913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:07.549913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:07.549913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:07.549913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:07.549913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:07.550807 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:07.550807 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:07.550807 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:07.550807 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:07.551024 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 384/754] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:07.551024 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ce944-4551-41e8-96ed-8ea76e0439a2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:07.553161 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 371/755] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:07.554862 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab3274-26a6-4708-ba86-7679f3f9ddb1 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:07.662570 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:07.663825 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab3274-26a6-4708-ba86-7679f3f9ddb1 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:07.682095 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ce944-4551-41e8-96ed-8ea76e0439a2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:07.711233 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdbcea4-9d0c-4a81-bf20-7c97863ac12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:07.711910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbcea4-9d0c-4a81-bf20-7c97863ac12d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:07.712631 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbcea4-9d0c-4a81-bf20-7c97863ac12d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:07.712927 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbcea4-9d0c-4a81-bf20-7c97863ac12d None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:07.713915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdbcea4-9d0c-4a81-bf20-7c97863ac12d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:07.714325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdbcea4-9d0c-4a81-bf20-7c97863ac12d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:07.716031 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdbcea4-9d0c-4a81-bf20-7c97863ac12d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:07.722217 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da96536-9550-4d4a-91ca-b0cf9ec53e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:07.723300 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da96536-9550-4d4a-91ca-b0cf9ec53e5b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:07.724265 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da96536-9550-4d4a-91ca-b0cf9ec53e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:07.724749 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da96536-9550-4d4a-91ca-b0cf9ec53e5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:07.732028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 372/756] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:07.733113 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdbcea4-9d0c-4a81-bf20-7c97863ac12d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:07.735419 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da96536-9550-4d4a-91ca-b0cf9ec53e5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:07.738351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da96536-9550-4d4a-91ca-b0cf9ec53e5b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:07.756380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5da96536-9550-4d4a-91ca-b0cf9ec53e5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:07.756380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:07.756380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:07.756380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:07.756380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:07.756380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:07.756380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:07.756380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:07.756380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:07.756380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:07.756380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:07.756380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:07.756380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:56:07.756380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:56:07.756380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:56:07.757398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:56:07.757398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:07.757398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:07.757398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:07.757398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:07.757398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:07.757398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:07.757398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:07.757398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:07.757398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:07.757398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:07.757398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:07.757398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:07.757398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:07.757398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:07.757398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:07.757398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:07.758299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:56:07.758299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:56:07.758299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:56:07.758299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:56:07.758299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:56:07.758299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:56:07.758299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:07.758299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:07.761014 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 385/757] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:56:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:07.761936 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da96536-9550-4d4a-91ca-b0cf9ec53e5b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:07.789705 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:07.867529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdbcea4-9d0c-4a81-bf20-7c97863ac12d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:07.889983 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da96536-9550-4d4a-91ca-b0cf9ec53e5b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:07.908739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3133c5-8924-4d64-abee-2ee847f3b76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:07.909446 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3133c5-8924-4d64-abee-2ee847f3b76f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:07.909733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3133c5-8924-4d64-abee-2ee847f3b76f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:07.910012 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3133c5-8924-4d64-abee-2ee847f3b76f None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:07.911037 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3133c5-8924-4d64-abee-2ee847f3b76f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:07.911352 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3133c5-8924-4d64-abee-2ee847f3b76f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:07.913000 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3133c5-8924-4d64-abee-2ee847f3b76f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:07.930192 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 373/758] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:07.931375 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3133c5-8924-4d64-abee-2ee847f3b76f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:07.935736 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d580c63-c38c-403a-b528-308c7498cff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:07.939301 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d580c63-c38c-403a-b528-308c7498cff2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:07.939301 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d580c63-c38c-403a-b528-308c7498cff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:07.939301 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d580c63-c38c-403a-b528-308c7498cff2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:07.947689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d580c63-c38c-403a-b528-308c7498cff2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:07.950017 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d580c63-c38c-403a-b528-308c7498cff2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:07.966302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3d580c63-c38c-403a-b528-308c7498cff2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:07.966302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:07.966302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:07.966302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:07.966302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:07.966302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:07.966302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:07.966302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:07.966302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:07.966302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:07.966302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:07.966302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:07.966302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:56:07.966302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:56:07.966302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:56:07.967212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:56:07.967212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:07.967212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:07.967212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:07.967212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:07.967212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:07.967212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:07.967212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:07.967212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:07.967212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:07.967212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:07.967212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:07.967212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:07.967212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:07.967212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:07.967212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:07.967212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:07.968163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:56:07.968163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:56:07.968163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:56:07.968163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:56:07.968163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:56:07.968163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:56:07.968163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:07.968163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:07.972200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 386/759] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:56:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:07.973052 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d580c63-c38c-403a-b528-308c7498cff2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:07.982982 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:08.051422 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:08.056638 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3133c5-8924-4d64-abee-2ee847f3b76f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:08.082684 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d580c63-c38c-403a-b528-308c7498cff2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:08.093484 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf2d35-0f8a-4d79-adc4-7d2ba868a307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:08.094466 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf2d35-0f8a-4d79-adc4-7d2ba868a307 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:08.094961 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf2d35-0f8a-4d79-adc4-7d2ba868a307 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:08.095443 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf2d35-0f8a-4d79-adc4-7d2ba868a307 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:08.096497 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf2d35-0f8a-4d79-adc4-7d2ba868a307 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:08.097034 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf2d35-0f8a-4d79-adc4-7d2ba868a307 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:08.098661 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf2d35-0f8a-4d79-adc4-7d2ba868a307 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:08.111467 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 374/760] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:56:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:08.112634 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf2d35-0f8a-4d79-adc4-7d2ba868a307 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:08.125618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fdfc7-ff9b-4265-a58f-f71665cf9435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:08.126729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fdfc7-ff9b-4265-a58f-f71665cf9435 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:08.127221 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fdfc7-ff9b-4265-a58f-f71665cf9435 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:08.127737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fdfc7-ff9b-4265-a58f-f71665cf9435 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:08.135963 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1fdfc7-ff9b-4265-a58f-f71665cf9435 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:08.137453 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1fdfc7-ff9b-4265-a58f-f71665cf9435 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:08.153831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ae1fdfc7-ff9b-4265-a58f-f71665cf9435 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:08.153831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:08.153831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:08.153831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:08.153831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:08.153831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:08.153831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:08.153831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:08.153831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:08.153831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:08.153831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:08.153831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:08.153831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:56:08.153831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:56:08.153831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:56:08.154729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:56:08.154729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:08.154729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:08.154729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:08.154729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:08.154729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:08.154729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:08.154729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:08.154729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:08.154729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:08.154729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:08.154729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:08.154729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:08.154729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:08.154729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:08.154729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:08.154729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:08.155632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:56:08.155632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:56:08.155632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:56:08.155632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:56:08.155632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:56:08.155632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:56:08.155632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:08.155632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:08.159934 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 387/761] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:56:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:08.160774 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fdfc7-ff9b-4265-a58f-f71665cf9435 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:08.238482 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:08.243915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf2d35-0f8a-4d79-adc4-7d2ba868a307 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:08.276908 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fdfc7-ff9b-4265-a58f-f71665cf9435 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:08.278708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32fd58-657b-4569-bae7-51124f474fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:08.280119 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32fd58-657b-4569-bae7-51124f474fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:08.280411 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32fd58-657b-4569-bae7-51124f474fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:08.280694 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32fd58-657b-4569-bae7-51124f474fe0 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:08.281483 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32fd58-657b-4569-bae7-51124f474fe0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:08.281797 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32fd58-657b-4569-bae7-51124f474fe0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:08.283227 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32fd58-657b-4569-bae7-51124f474fe0 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:08.301210 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 375/762] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:56:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:08.301210 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32fd58-657b-4569-bae7-51124f474fe0 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:08.316844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b0146-d5cd-4590-ac59-f0fb8cd96aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:08.317501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b0146-d5cd-4590-ac59-f0fb8cd96aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:08.317906 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b0146-d5cd-4590-ac59-f0fb8cd96aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:08.318191 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b0146-d5cd-4590-ac59-f0fb8cd96aa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:08.319028 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b0146-d5cd-4590-ac59-f0fb8cd96aa0 None admin] RBAC: Adding query filter params (role.id=5c4ba827ac854bc69cc26c3bcd27dc5c, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=94d824bffc2c4af299c537056bc21734) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:08.319346 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b0146-d5cd-4590-ac59-f0fb8cd96aa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:08.320935 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2b0146-d5cd-4590-ac59-f0fb8cd96aa0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:08.322022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:08.327297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 388/763] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:56:08 2022] GET /identity/v3/role_assignments?user.id=94d824bffc2c4af299c537056bc21734&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=5c4ba827ac854bc69cc26c3bcd27dc5c => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:08.328562 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc64fef-0784-4be1-b644-52035291b276 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:08.328858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc64fef-0784-4be1-b644-52035291b276 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:08.329390 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc64fef-0784-4be1-b644-52035291b276 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:08.335034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 389/764] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:08.335034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc64fef-0784-4be1-b644-52035291b276 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:08.427177 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32fd58-657b-4569-bae7-51124f474fe0 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:08.457459 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc64fef-0784-4be1-b644-52035291b276 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:08.465343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face0068-f96f-4b32-b7b9-4f1c5db17392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:08.466081 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face0068-f96f-4b32-b7b9-4f1c5db17392 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:08.466370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face0068-f96f-4b32-b7b9-4f1c5db17392 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:08.466701 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face0068-f96f-4b32-b7b9-4f1c5db17392 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:08.467516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face0068-f96f-4b32-b7b9-4f1c5db17392 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:08.467832 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face0068-f96f-4b32-b7b9-4f1c5db17392 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:08.469415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face0068-f96f-4b32-b7b9-4f1c5db17392 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:08.484129 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 376/765] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:56:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:08.485999 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9979870-24d1-4227-b595-bc47b528422e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:08.486515 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9979870-24d1-4227-b595-bc47b528422e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:08.487000 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9979870-24d1-4227-b595-bc47b528422e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:08.490981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:08.499461 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae71dce-1b46-41fd-a9cc-c9b1167b7ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:08.500417 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae71dce-1b46-41fd-a9cc-c9b1167b7ec6 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:08.500705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae71dce-1b46-41fd-a9cc-c9b1167b7ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:08.500981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae71dce-1b46-41fd-a9cc-c9b1167b7ec6 None admin] PATH_INFO: `/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/ac717fcd9fc1499db12cf852d45572a8/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:08.503968 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae71dce-1b46-41fd-a9cc-c9b1167b7ec6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec94946acf91486ba3f5b4fe834b8076, user_id=ac717fcd9fc1499db12cf852d45572a8, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:08.505434 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae71dce-1b46-41fd-a9cc-c9b1167b7ec6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:08.508215 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b9979870-24d1-4227-b595-bc47b528422e None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:08.509427 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b9979870-24d1-4227-b595-bc47b528422e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:08.526280 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 390/766] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:56:08 2022] PUT /identity/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/ac717fcd9fc1499db12cf852d45572a8/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:56:08.526857 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae71dce-1b46-41fd-a9cc-c9b1167b7ec6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:08.601859 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 377/767] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:08.642354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9979870-24d1-4227-b595-bc47b528422e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:08.661870 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:08.741735 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae71dce-1b46-41fd-a9cc-c9b1167b7ec6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:08.794239 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a92fa7-bfd2-4f19-8dfd-918422c202a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:08.794860 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9979870-24d1-4227-b595-bc47b528422e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:08.794978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a92fa7-bfd2-4f19-8dfd-918422c202a2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:08.795196 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a92fa7-bfd2-4f19-8dfd-918422c202a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:08.795479 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a92fa7-bfd2-4f19-8dfd-918422c202a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:08.796476 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a92fa7-bfd2-4f19-8dfd-918422c202a2 None admin] RBAC: Adding query filter params (role.id=c60a99de086f45c6820f7872a6b43feb, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=cbf513ee25374e49b3d8ce28843e393a) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:08.796797 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a92fa7-bfd2-4f19-8dfd-918422c202a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:08.798371 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a92fa7-bfd2-4f19-8dfd-918422c202a2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:08.805704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 391/768] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:56:08 2022] GET /identity/v3/role_assignments?user.id=cbf513ee25374e49b3d8ce28843e393a&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=c60a99de086f45c6820f7872a6b43feb => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:08.843617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbfd5a4-7bc2-4a37-802d-bfd257f51f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:08.844479 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfd5a4-7bc2-4a37-802d-bfd257f51f71 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:08.844765 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfd5a4-7bc2-4a37-802d-bfd257f51f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:08.845054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfd5a4-7bc2-4a37-802d-bfd257f51f71 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:08.845732 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbfd5a4-7bc2-4a37-802d-bfd257f51f71 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:08.847160 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbfd5a4-7bc2-4a37-802d-bfd257f51f71 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:08.867622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-7dbfd5a4-7bc2-4a37-802d-bfd257f51f71 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 13:56:08.867622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:08.867622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:56:08.867622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:56:08.867622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:56:08.867622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:56:08.867622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:56:08.867622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:08.867622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 13:56:08.867622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:08.867622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:08.867622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:08.867622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:08.867622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:08.867622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:08.867622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:08.868851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:08.868851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:08.868851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:08.868851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:08.868851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:08.868851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:08.868851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:08.868851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:56:08.868851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:56:08.868851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:56:08.868851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:56:08.868851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:08.868851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:08.868851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:08.868851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:08.868851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:08.869916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:08.869916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:08.869916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:08.869916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:08.869916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:08.869916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:08.869916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:08.869916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:08.869916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:08.869916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:08.869916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:08.869916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:56:08.869916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:08.869916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 13:56:08.869916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:08.869916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 378/769] 10.222.0.94 () {56 vars in 1129 bytes} [Mon Jul 18 13:56:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:08.877765 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a92fa7-bfd2-4f19-8dfd-918422c202a2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:08.973605 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:09.035850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a92fa7-bfd2-4f19-8dfd-918422c202a2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:09.040408 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cf52d-4d12-4106-a7e6-329120eae78b None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:09.040707 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cf52d-4d12-4106-a7e6-329120eae78b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:09.040994 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cf52d-4d12-4106-a7e6-329120eae78b None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:09.043662 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 379/770] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:09.047868 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d983841-f732-4945-b847-688bc34ade7c None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:09.048197 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d983841-f732-4945-b847-688bc34ade7c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:09.048477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d983841-f732-4945-b847-688bc34ade7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:09.066324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-7d983841-f732-4945-b847-688bc34ade7c None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:09.067517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7d983841-f732-4945-b847-688bc34ade7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:09.080112 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0f819-1e61-4390-9b3e-32bb7da58bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:09.082195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0f819-1e61-4390-9b3e-32bb7da58bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:09.082423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0f819-1e61-4390-9b3e-32bb7da58bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:09.082715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0f819-1e61-4390-9b3e-32bb7da58bc3 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:09.083425 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0f819-1e61-4390-9b3e-32bb7da58bc3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:09.083743 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0f819-1e61-4390-9b3e-32bb7da58bc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:09.087726 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0f819-1e61-4390-9b3e-32bb7da58bc3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:09.097762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 392/771] 10.222.0.94 () {56 vars in 1124 bytes} [Mon Jul 18 13:56:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:09.103850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0f819-1e61-4390-9b3e-32bb7da58bc3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:09.175968 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 380/772] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 13:56:09.221451 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d983841-f732-4945-b847-688bc34ade7c None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:09.225058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:09.270769 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0f819-1e61-4390-9b3e-32bb7da58bc3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:09.308442 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:09.341847 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b94c1-9f62-4fe6-808a-14ae0cf99f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:09.342857 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b94c1-9f62-4fe6-808a-14ae0cf99f0c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:09.344093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b94c1-9f62-4fe6-808a-14ae0cf99f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:09.344556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b94c1-9f62-4fe6-808a-14ae0cf99f0c None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:09.348852 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b94c1-9f62-4fe6-808a-14ae0cf99f0c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:09.349219 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b94c1-9f62-4fe6-808a-14ae0cf99f0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:09.350791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b94c1-9f62-4fe6-808a-14ae0cf99f0c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:09.362863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 393/773] 10.222.0.94 () {56 vars in 1124 bytes} [Mon Jul 18 13:56:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:09.365069 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b94c1-9f62-4fe6-808a-14ae0cf99f0c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:09.426541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d983841-f732-4945-b847-688bc34ade7c None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:09.465825 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4135d5-a97f-44d6-973a-6d1a07c0061b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:09.466641 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4135d5-a97f-44d6-973a-6d1a07c0061b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:09.466933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4135d5-a97f-44d6-973a-6d1a07c0061b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:09.467230 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4135d5-a97f-44d6-973a-6d1a07c0061b None admin] PATH_INFO: `/v3/services/network` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:09.469862 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4135d5-a97f-44d6-973a-6d1a07c0061b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:09.469862 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4135d5-a97f-44d6-973a-6d1a07c0061b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:09.483308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-3d4135d5-a97f-44d6-973a-6d1a07c0061b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 13:56:09.483308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:09.483308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:56:09.483308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:56:09.483308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:56:09.483308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:56:09.483308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:56:09.483308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:09.483308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 13:56:09.483308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:09.483308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:09.483308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:09.483308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:09.483308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:09.483308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:09.483308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:09.483308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:09.484352 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:09.484352 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:09.484352 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:09.484352 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:09.484352 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:09.484352 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:09.484352 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:56:09.484352 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:56:09.484352 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:56:09.484352 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:56:09.484352 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:09.484352 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:09.484352 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:09.484352 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:09.484352 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:09.484352 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:09.485453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:09.485453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:09.485453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:09.485453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:09.485453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:09.485453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:09.485453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:09.485453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:09.485453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:09.485453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:09.485453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:56:09.485453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:09.485453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 13:56:09.485453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:09.485453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 381/774] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:56:09 2022] GET /identity/v3/services/network => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:09.492316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4135d5-a97f-44d6-973a-6d1a07c0061b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:09.566050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:09.597793 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b94c1-9f62-4fe6-808a-14ae0cf99f0c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:09.603050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4135d5-a97f-44d6-973a-6d1a07c0061b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:09.639625 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f90b5-08a6-4c71-acfe-a3ccb480e398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:09.640733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f90b5-08a6-4c71-acfe-a3ccb480e398 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:09.641232 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f90b5-08a6-4c71-acfe-a3ccb480e398 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:09.641720 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f90b5-08a6-4c71-acfe-a3ccb480e398 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:09.642632 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f90b5-08a6-4c71-acfe-a3ccb480e398 None admin] RBAC: Adding query filter params (name=network) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:09.643147 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f90b5-08a6-4c71-acfe-a3ccb480e398 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:09.644763 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f90b5-08a6-4c71-acfe-a3ccb480e398 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:09.650673 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 382/775] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Jul 18 13:56:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:09.660671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f90b5-08a6-4c71-acfe-a3ccb480e398 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:09.680125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda3cf6-23b0-4d03-8d09-81a1d708f95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:09.680840 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda3cf6-23b0-4d03-8d09-81a1d708f95b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:09.681132 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda3cf6-23b0-4d03-8d09-81a1d708f95b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:09.681415 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda3cf6-23b0-4d03-8d09-81a1d708f95b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:09.682222 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda3cf6-23b0-4d03-8d09-81a1d708f95b None admin] RBAC: Adding query filter params (interface=public, service_id=3385957fd8c44ff2af310a75cddf3d2d, region_id=RegionOne) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:09.682547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda3cf6-23b0-4d03-8d09-81a1d708f95b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:09.683958 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda3cf6-23b0-4d03-8d09-81a1d708f95b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:09.710902 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 394/776] 10.222.0.94 () {56 vars in 1251 bytes} [Mon Jul 18 13:56:09 2022] GET /identity/v3/endpoints?service_id=3385957fd8c44ff2af310a75cddf3d2d&interface=public®ion_id=RegionOne => generated 194 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:09.715813 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda3cf6-23b0-4d03-8d09-81a1d708f95b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:09.771399 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f90b5-08a6-4c71-acfe-a3ccb480e398 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:09.808311 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:09.811235 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a87ef6-8690-4079-b7e6-c9ff2e9caedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:09.813065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a87ef6-8690-4079-b7e6-c9ff2e9caedc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:09.813367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a87ef6-8690-4079-b7e6-c9ff2e9caedc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:09.813659 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a87ef6-8690-4079-b7e6-c9ff2e9caedc None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:09.814384 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a87ef6-8690-4079-b7e6-c9ff2e9caedc None admin] RBAC: Adding query filter params (type=network) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:09.814709 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a87ef6-8690-4079-b7e6-c9ff2e9caedc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:09.816188 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a87ef6-8690-4079-b7e6-c9ff2e9caedc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:09.826448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 383/777] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Jul 18 13:56:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:09.833636 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a87ef6-8690-4079-b7e6-c9ff2e9caedc None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:09.849341 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:09.993112 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a87ef6-8690-4079-b7e6-c9ff2e9caedc None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:10.001856 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda3cf6-23b0-4d03-8d09-81a1d708f95b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:10.028231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9defad1b-0897-4196-acd0-ce5c32aa1189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:10.028898 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9defad1b-0897-4196-acd0-ce5c32aa1189 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:10.029189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9defad1b-0897-4196-acd0-ce5c32aa1189 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:10.029476 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9defad1b-0897-4196-acd0-ce5c32aa1189 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:10.030267 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9defad1b-0897-4196-acd0-ce5c32aa1189 None admin] RBAC: Adding query filter params (interface=public, service_id=7fb4fb7e9953476a83a915ccde16090c, region_id=RegionOne) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:10.030586 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9defad1b-0897-4196-acd0-ce5c32aa1189 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:10.032102 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9defad1b-0897-4196-acd0-ce5c32aa1189 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:10.040562 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 384/778] 10.222.0.94 () {56 vars in 1251 bytes} [Mon Jul 18 13:56:09 2022] GET /identity/v3/endpoints?service_id=7fb4fb7e9953476a83a915ccde16090c&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:10.049609 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9defad1b-0897-4196-acd0-ce5c32aa1189 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:10.067262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe827d99-7020-494b-b3c5-f0a745f6fb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:10.067875 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe827d99-7020-494b-b3c5-f0a745f6fb77 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:10.068193 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe827d99-7020-494b-b3c5-f0a745f6fb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:10.070843 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe827d99-7020-494b-b3c5-f0a745f6fb77 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:10.071542 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe827d99-7020-494b-b3c5-f0a745f6fb77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:10.072953 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe827d99-7020-494b-b3c5-f0a745f6fb77 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:10.082858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 395/779] 10.222.0.94 () {56 vars in 1087 bytes} [Mon Jul 18 13:56:09 2022] GET /identity/v3/services => generated 772 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:10.141501 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:10.187715 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9defad1b-0897-4196-acd0-ce5c32aa1189 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:10.223907 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb3a4f-8303-4484-960c-11978ac81224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:10.224624 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb3a4f-8303-4484-960c-11978ac81224 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:10.224911 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb3a4f-8303-4484-960c-11978ac81224 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:10.225191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb3a4f-8303-4484-960c-11978ac81224 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:10.226422 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb3a4f-8303-4484-960c-11978ac81224 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:10.227808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb3a4f-8303-4484-960c-11978ac81224 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:10.236941 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 385/780] 10.222.0.94 () {56 vars in 1087 bytes} [Mon Jul 18 13:56:10 2022] GET /identity/v3/services => generated 772 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:10.753607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4fb48-2585-4ca6-963a-91b7d30469a1 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:10.754311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4fb48-2585-4ca6-963a-91b7d30469a1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:10.754801 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4fb48-2585-4ca6-963a-91b7d30469a1 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:10.756767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 396/781] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:10.762454 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92637b8-a63c-4a8f-9a0b-866967808a1d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:10.763347 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92637b8-a63c-4a8f-9a0b-866967808a1d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:10.763828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92637b8-a63c-4a8f-9a0b-866967808a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:10.782255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b92637b8-a63c-4a8f-9a0b-866967808a1d None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:10.783978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b92637b8-a63c-4a8f-9a0b-866967808a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:10.868819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 386/782] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:10.915341 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4fb48-2585-4ca6-963a-91b7d30469a1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:10.930749 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bb40f-3719-49bd-8e28-8d3a56a9c753 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:10.931056 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bb40f-3719-49bd-8e28-8d3a56a9c753 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:10.931329 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bb40f-3719-49bd-8e28-8d3a56a9c753 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:10.933113 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 387/783] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:56:10.950119 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c50ad9-ac68-412f-96b1-bd268ee229b0 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:10.950649 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c50ad9-ac68-412f-96b1-bd268ee229b0 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:10.951135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c50ad9-ac68-412f-96b1-bd268ee229b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:10.974172 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-38c50ad9-ac68-412f-96b1-bd268ee229b0 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:10.976628 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-38c50ad9-ac68-412f-96b1-bd268ee229b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:11.081678 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:11.087843 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4fb48-2585-4ca6-963a-91b7d30469a1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:11.108352 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:11.123509 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 388/784] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 13:56:11.152588 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce094a9-dbc2-428d-9424-be2a2a6e6574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:11.153803 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce094a9-dbc2-428d-9424-be2a2a6e6574 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:11.154297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce094a9-dbc2-428d-9424-be2a2a6e6574 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:11.154778 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce094a9-dbc2-428d-9424-be2a2a6e6574 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:11.167658 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce094a9-dbc2-428d-9424-be2a2a6e6574 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:56:11.175920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6ce094a9-dbc2-428d-9424-be2a2a6e6574 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:56:11.175920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:11.175920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:11.175920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:11.175920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:11.175920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:11.175920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:11.175920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:11.175920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:11.175920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:11.175920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:11.175920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:11.175920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:56:11.175920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:56:11.175920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:56:11.175920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:56:11.176911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:56:11.176911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:56:11.176911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:56:11.176911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:56:11.176911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:11.176911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:11.176911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:11.176911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:11.176911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:11.176911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:11.176911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:11.176911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:11.176911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:11.176911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:11.176911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:11.176911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:11.177776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:11.177776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:11.177776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:11.177776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:11.177776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:56:11.177776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:56:11.177776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:56:11.177776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:56:11.177776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:56:11.177776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:56:11.177776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:56:11.177776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:11.181257 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 397/785] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:56:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:11.189054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c50ad9-ac68-412f-96b1-bd268ee229b0 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:11.198089 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce094a9-dbc2-428d-9424-be2a2a6e6574 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:11.352440 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c50ad9-ac68-412f-96b1-bd268ee229b0 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:11.361822 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce094a9-dbc2-428d-9424-be2a2a6e6574 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:11.390726 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48c186-f1ee-4f14-a35f-85683f2e6af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:11.391392 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48c186-f1ee-4f14-a35f-85683f2e6af9 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:11.391678 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48c186-f1ee-4f14-a35f-85683f2e6af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:11.391973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48c186-f1ee-4f14-a35f-85683f2e6af9 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:11.392731 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48c186-f1ee-4f14-a35f-85683f2e6af9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:11.393034 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48c186-f1ee-4f14-a35f-85683f2e6af9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:11.394403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48c186-f1ee-4f14-a35f-85683f2e6af9 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:11.402730 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 389/786] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Jul 18 13:56:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:11.403776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c35b7-3f4d-4a39-a0d7-1033b867881a None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:11.404099 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c35b7-3f4d-4a39-a0d7-1033b867881a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:11.404381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c35b7-3f4d-4a39-a0d7-1033b867881a None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:11.407429 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c658e-ef3e-4e98-b178-b40589399588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:11.408568 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c658e-ef3e-4e98-b178-b40589399588 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:11.409063 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c658e-ef3e-4e98-b178-b40589399588 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:11.409626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c658e-ef3e-4e98-b178-b40589399588 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:11.409817 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 390/787] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:11.410570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c35b7-3f4d-4a39-a0d7-1033b867881a None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:11.423448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69c658e-ef3e-4e98-b178-b40589399588 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:56:11.432183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a69c658e-ef3e-4e98-b178-b40589399588 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:56:11.432183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:11.432183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:11.432183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:11.432183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:11.432183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:11.432183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:11.432183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:11.432183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:11.432183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:11.432183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:11.432183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:11.432183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:56:11.432183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:56:11.432183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:56:11.432183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:56:11.433131 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:56:11.433131 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:56:11.433131 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:56:11.433131 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:56:11.433131 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:11.433131 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:11.433131 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:11.433131 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:11.433131 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:11.433131 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:11.433131 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:11.433131 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:11.433131 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:11.433131 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:11.433131 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:11.433131 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:11.433975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:11.433975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:11.433975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:11.433975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:11.433975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:56:11.433975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:56:11.433975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:56:11.433975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:56:11.433975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:56:11.433975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:56:11.433975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:56:11.433975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:11.433975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 398/788] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:56:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:11.435179 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1e4ae-1e41-447b-a32b-f10145869f2d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:11.435179 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1e4ae-1e41-447b-a32b-f10145869f2d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:11.435345 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de1e4ae-1e41-447b-a32b-f10145869f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:11.453462 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3de1e4ae-1e41-447b-a32b-f10145869f2d None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:11.454917 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3de1e4ae-1e41-447b-a32b-f10145869f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:11.479304 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:11.479775 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:11.544014 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 399/789] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:11.545215 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de1e4ae-1e41-447b-a32b-f10145869f2d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:11.548175 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c35b7-3f4d-4a39-a0d7-1033b867881a None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:11.588301 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd2fd2-4331-4fca-a624-c0483059bc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:11.589209 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd2fd2-4331-4fca-a624-c0483059bc6c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:11.589497 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd2fd2-4331-4fca-a624-c0483059bc6c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:11.589799 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd2fd2-4331-4fca-a624-c0483059bc6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:11.602569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd2fd2-4331-4fca-a624-c0483059bc6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:11.603971 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd2fd2-4331-4fca-a624-c0483059bc6c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:11.623858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-89cd2fd2-4331-4fca-a624-c0483059bc6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:11.623858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:11.623858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:11.623858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:11.623858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:11.623858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:11.623858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:11.623858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:11.623858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:11.623858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:11.623858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:11.623858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:11.623858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:11.623858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:11.623858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:11.624900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:11.624900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:11.624900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:11.624900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:11.624900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:11.624900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:11.624900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:11.624900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:11.624900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:11.624900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:11.624900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:11.624900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:11.624900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:11.624900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:11.624900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:11.624900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:11.624900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:11.625845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:11.625845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:11.625845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:11.625845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:11.625845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 391/790] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:11.626654 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd2fd2-4331-4fca-a624-c0483059bc6c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:11.672681 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de1e4ae-1e41-447b-a32b-f10145869f2d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:11.709362 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbafbd-c3cc-4d26-b9ef-a4fc0add3c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:11.710471 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbafbd-c3cc-4d26-b9ef-a4fc0add3c97 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:11.710981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbafbd-c3cc-4d26-b9ef-a4fc0add3c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:11.711457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbafbd-c3cc-4d26-b9ef-a4fc0add3c97 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:11.712431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcbafbd-c3cc-4d26-b9ef-a4fc0add3c97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:11.712938 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcbafbd-c3cc-4d26-b9ef-a4fc0add3c97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:11.714509 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcbafbd-c3cc-4d26-b9ef-a4fc0add3c97 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:11.723234 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 400/791] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:56:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:11.724366 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbafbd-c3cc-4d26-b9ef-a4fc0add3c97 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:11.736213 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:11.741192 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd2fd2-4331-4fca-a624-c0483059bc6c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:11.750951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:11.780254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c53ad-98bd-441c-a23f-c2a261ad66ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:11.781132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c53ad-98bd-441c-a23f-c2a261ad66ff None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:11.781433 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c53ad-98bd-441c-a23f-c2a261ad66ff None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:11.781717 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c53ad-98bd-441c-a23f-c2a261ad66ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:11.793620 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c53ad-98bd-441c-a23f-c2a261ad66ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:56:11.802331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-cc8c53ad-98bd-441c-a23f-c2a261ad66ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:56:11.802331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:11.802331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:11.802331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:11.802331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:11.802331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:11.802331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:11.802331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:11.802331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:11.802331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:11.802331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:11.802331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:11.802331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:56:11.802331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:56:11.802331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:56:11.802331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:56:11.803349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:56:11.803349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:56:11.803349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:56:11.803349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:56:11.803349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:11.803349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:11.803349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:11.803349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:11.803349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:11.803349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:11.803349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:11.803349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:11.803349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:11.803349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:11.803349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:11.803349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:11.804325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:11.804325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:11.804325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:11.804325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:11.804325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:56:11.804325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:56:11.804325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:56:11.804325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:56:11.804325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:56:11.804325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:56:11.804325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:56:11.804325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:11.808086 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 392/792] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:56:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:11.808966 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c53ad-98bd-441c-a23f-c2a261ad66ff None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:11.837289 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbafbd-c3cc-4d26-b9ef-a4fc0add3c97 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:11.873314 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85362cb-09af-45d8-b998-a022ae871024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:11.874279 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85362cb-09af-45d8-b998-a022ae871024 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:11.874773 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85362cb-09af-45d8-b998-a022ae871024 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:11.875332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85362cb-09af-45d8-b998-a022ae871024 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:11.876363 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85362cb-09af-45d8-b998-a022ae871024 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:11.876870 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85362cb-09af-45d8-b998-a022ae871024 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:11.878471 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85362cb-09af-45d8-b998-a022ae871024 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:11.891875 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 401/793] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:11.893030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85362cb-09af-45d8-b998-a022ae871024 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:11.917975 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c53ad-98bd-441c-a23f-c2a261ad66ff None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:11.965793 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816c022-6c97-4323-87fa-26547ecd5a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:11.966598 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816c022-6c97-4323-87fa-26547ecd5a34 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:11.966892 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816c022-6c97-4323-87fa-26547ecd5a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:11.967176 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816c022-6c97-4323-87fa-26547ecd5a34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:11.978580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816c022-6c97-4323-87fa-26547ecd5a34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:11.980058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816c022-6c97-4323-87fa-26547ecd5a34 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:11.980984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:11.999258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-6816c022-6c97-4323-87fa-26547ecd5a34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:11.999258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:11.999258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:11.999258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:11.999258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:11.999258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:11.999258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:11.999258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:11.999258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:11.999258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:11.999258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:11.999258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:11.999258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:11.999258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:11.999258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:12.000360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:12.000360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:12.000360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:12.000360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:12.000360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:12.000360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:12.000360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:12.000360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:12.000360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:12.000360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:12.000360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:12.000360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:12.000360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:12.000360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:12.000360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:12.000360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:12.000360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:12.001285 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:12.001285 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:12.001285 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:12.001285 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:12.001515 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85362cb-09af-45d8-b998-a022ae871024 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:12.003657 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 393/794] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:12.004687 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816c022-6c97-4323-87fa-26547ecd5a34 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:12.018397 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:12.040536 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0388725-17fd-45fe-adbf-cbf13498d797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:12.041311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0388725-17fd-45fe-adbf-cbf13498d797 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:12.041674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0388725-17fd-45fe-adbf-cbf13498d797 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:12.042023 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0388725-17fd-45fe-adbf-cbf13498d797 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:12.042937 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0388725-17fd-45fe-adbf-cbf13498d797 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:12.043314 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0388725-17fd-45fe-adbf-cbf13498d797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:12.044917 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0388725-17fd-45fe-adbf-cbf13498d797 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:12.055990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 402/795] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:56:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:12.055990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0388725-17fd-45fe-adbf-cbf13498d797 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:12.117667 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816c022-6c97-4323-87fa-26547ecd5a34 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:12.154116 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572efcbd-f2f6-4726-b52f-5f90c3a37e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:12.155042 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572efcbd-f2f6-4726-b52f-5f90c3a37e3c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:12.155406 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572efcbd-f2f6-4726-b52f-5f90c3a37e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:12.155760 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572efcbd-f2f6-4726-b52f-5f90c3a37e3c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:12.162334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0388725-17fd-45fe-adbf-cbf13498d797 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:12.164090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572efcbd-f2f6-4726-b52f-5f90c3a37e3c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:12.165639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572efcbd-f2f6-4726-b52f-5f90c3a37e3c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:12.182495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-572efcbd-f2f6-4726-b52f-5f90c3a37e3c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 13:56:12.182495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:12.182495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:12.182495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:12.182495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:12.182495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:12.182495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:12.182495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:12.182495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:12.182495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:12.182495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:12.182495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:12.182495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:56:12.182495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:56:12.182495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:56:12.182495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:56:12.183670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:12.183670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:12.183670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:56:12.183670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:12.183670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:56:12.183670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:12.183670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:12.183670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:12.183670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:12.183670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:12.183670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:12.183670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:12.183670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:12.183670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:12.183670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:12.183670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:12.183670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:12.183670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:12.184886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:12.184886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:12.184886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:56:12.184886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:56:12.184886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:56:12.184886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:56:12.184886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:56:12.184886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:56:12.184886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 13:56:12.184886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:12.186363 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 394/796] 10.222.0.94 () {56 vars in 1117 bytes} [Mon Jul 18 13:56:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:12.187307 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572efcbd-f2f6-4726-b52f-5f90c3a37e3c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:12.200299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011af54-1159-43b2-a441-156d5def2a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:12.201090 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011af54-1159-43b2-a441-156d5def2a17 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:12.201382 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011af54-1159-43b2-a441-156d5def2a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:12.201663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011af54-1159-43b2-a441-156d5def2a17 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:12.202395 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011af54-1159-43b2-a441-156d5def2a17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:12.202703 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011af54-1159-43b2-a441-156d5def2a17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:12.204487 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011af54-1159-43b2-a441-156d5def2a17 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:12.217874 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 403/797] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:12.218782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011af54-1159-43b2-a441-156d5def2a17 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:12.222742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:12.275260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:12.318473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572efcbd-f2f6-4726-b52f-5f90c3a37e3c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:12.343725 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011af54-1159-43b2-a441-156d5def2a17 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:12.357129 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87c734-3265-48d9-8c5e-8f6f2b44f538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:12.358132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87c734-3265-48d9-8c5e-8f6f2b44f538 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:12.358523 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87c734-3265-48d9-8c5e-8f6f2b44f538 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:12.359002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87c734-3265-48d9-8c5e-8f6f2b44f538 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:12.370780 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b87c734-3265-48d9-8c5e-8f6f2b44f538 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:12.372371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b87c734-3265-48d9-8c5e-8f6f2b44f538 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:12.383632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1c873-6753-41e4-ab0e-c17a3a32e5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:12.384572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1c873-6753-41e4-ab0e-c17a3a32e5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:12.385019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1c873-6753-41e4-ab0e-c17a3a32e5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:12.385448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1c873-6753-41e4-ab0e-c17a3a32e5e6 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:12.386424 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff1c873-6753-41e4-ab0e-c17a3a32e5e6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:12.386878 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff1c873-6753-41e4-ab0e-c17a3a32e5e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:12.388504 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff1c873-6753-41e4-ab0e-c17a3a32e5e6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:12.391024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-3b87c734-3265-48d9-8c5e-8f6f2b44f538 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:12.391024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:12.391024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:12.391024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:12.391024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:12.391024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:12.391024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:12.391024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:12.391024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:12.391024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:12.391024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:12.391024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:12.391024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:12.391024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:12.391024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:12.392074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:12.392074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:12.392074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:12.392074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:12.392074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:12.392074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:12.392074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:12.392074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:12.392074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:12.392074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:12.392074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:12.392074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:12.392074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:12.392074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:12.392074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:12.392074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:12.392074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:12.392982 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:12.392982 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:12.392982 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:12.392982 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:12.397911 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 395/798] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:12.398859 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87c734-3265-48d9-8c5e-8f6f2b44f538 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:12.410832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 404/799] 10.222.0.94 () {56 vars in 1149 bytes} [Mon Jul 18 13:56:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:12.412040 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1c873-6753-41e4-ab0e-c17a3a32e5e6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:12.479109 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:12.504192 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87c734-3265-48d9-8c5e-8f6f2b44f538 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:12.522863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1c873-6753-41e4-ab0e-c17a3a32e5e6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:12.528036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:12.541504 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d8676f-e753-4204-b6da-b546bdd3bf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:12.542309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8676f-e753-4204-b6da-b546bdd3bf1b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:12.542599 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8676f-e753-4204-b6da-b546bdd3bf1b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:12.542881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8676f-e753-4204-b6da-b546bdd3bf1b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:12.550530 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d8676f-e753-4204-b6da-b546bdd3bf1b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:12.552003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d8676f-e753-4204-b6da-b546bdd3bf1b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:12.559991 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30ae45-69f1-4043-aa96-564dab11307e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:12.560661 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30ae45-69f1-4043-aa96-564dab11307e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:12.561009 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30ae45-69f1-4043-aa96-564dab11307e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:12.561296 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30ae45-69f1-4043-aa96-564dab11307e None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:12.562018 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30ae45-69f1-4043-aa96-564dab11307e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:12.562329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30ae45-69f1-4043-aa96-564dab11307e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:12.563697 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30ae45-69f1-4043-aa96-564dab11307e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:12.566365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-59d8676f-e753-4204-b6da-b546bdd3bf1b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 13:56:12.566365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:12.566365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:12.566365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:12.566365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:12.566365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:12.566365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:12.566365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:12.566365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:12.566365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:12.566365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:12.566365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:12.566365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:56:12.566365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:56:12.566365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:56:12.566365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:56:12.567282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:12.567282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:12.567282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:56:12.567282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:12.567282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:56:12.567282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:12.567282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:12.567282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:12.567282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:12.567282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:12.567282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:12.567282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:12.567282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:12.567282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:12.567282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:12.567282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:12.567282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:12.567282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:12.568258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:12.568258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:12.568258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:56:12.568258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:56:12.568258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:56:12.568258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:56:12.568258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:56:12.568258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:56:12.568258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 13:56:12.568258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:12.568258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 396/800] 10.222.0.94 () {56 vars in 1093 bytes} [Mon Jul 18 13:56:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:12.568867 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d8676f-e753-4204-b6da-b546bdd3bf1b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:12.576330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 405/801] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:12.577749 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbc0db-17d5-4b09-a7ef-c3ff66f29d88 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:12.578048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbc0db-17d5-4b09-a7ef-c3ff66f29d88 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:12.578338 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbc0db-17d5-4b09-a7ef-c3ff66f29d88 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:12.579823 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 406/802] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:12.581218 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac57aec-e5f8-41f2-be2e-a36404b30e21 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:12.581566 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac57aec-e5f8-41f2-be2e-a36404b30e21 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:12.581856 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac57aec-e5f8-41f2-be2e-a36404b30e21 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:12.583796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 407/803] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:12.584272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac57aec-e5f8-41f2-be2e-a36404b30e21 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:12.671873 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d8676f-e753-4204-b6da-b546bdd3bf1b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:12.685444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac57aec-e5f8-41f2-be2e-a36404b30e21 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:12.704835 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274de5b5-3c9f-463b-90fa-e502578be526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:12.705623 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274de5b5-3c9f-463b-90fa-e502578be526 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:12.705914 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274de5b5-3c9f-463b-90fa-e502578be526 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:12.706198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274de5b5-3c9f-463b-90fa-e502578be526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:12.707694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:12.717630 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274de5b5-3c9f-463b-90fa-e502578be526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:12.719082 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274de5b5-3c9f-463b-90fa-e502578be526 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:12.720688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75972c-bacf-449d-9886-18a431749d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:12.721352 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75972c-bacf-449d-9886-18a431749d9f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:12.721646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75972c-bacf-449d-9886-18a431749d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:12.721926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75972c-bacf-449d-9886-18a431749d9f None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:12.722757 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75972c-bacf-449d-9886-18a431749d9f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:12.723071 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75972c-bacf-449d-9886-18a431749d9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:12.724514 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd75972c-bacf-449d-9886-18a431749d9f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:12.736771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-274de5b5-3c9f-463b-90fa-e502578be526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:12.736771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:12.736771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:12.736771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:12.736771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:12.736771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:12.736771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:12.736771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:12.736771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:12.736771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:12.736771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:12.736771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:12.736771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:12.736771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:12.736771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:12.737815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:12.737815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:12.737815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:12.737815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:12.737815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:12.737815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:12.737815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:12.737815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:12.737815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:12.737815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:12.737815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:12.737815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:12.737815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:12.737815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:12.737815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:12.737815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:12.737815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:12.738770 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:12.738770 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:12.738770 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:12.738770 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:12.738770 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 397/804] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:12.739101 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274de5b5-3c9f-463b-90fa-e502578be526 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:12.743363 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 408/805] 10.222.0.94 () {56 vars in 1133 bytes} [Mon Jul 18 13:56:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:12.744644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ea0e8-adde-4a0a-bc8a-ff0e0b33ea90 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:12.744941 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ea0e8-adde-4a0a-bc8a-ff0e0b33ea90 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:12.745224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ea0e8-adde-4a0a-bc8a-ff0e0b33ea90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:12.761497 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bd1ea0e8-adde-4a0a-bc8a-ff0e0b33ea90 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:12.762688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bd1ea0e8-adde-4a0a-bc8a-ff0e0b33ea90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:12.774176 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:12.828635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 409/806] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:12.830152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7760c4-108e-4f9f-a106-cf2076c18902 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:12.830448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7760c4-108e-4f9f-a106-cf2076c18902 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:12.830732 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7760c4-108e-4f9f-a106-cf2076c18902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:12.846496 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cb7760c4-108e-4f9f-a106-cf2076c18902 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:12.847688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cb7760c4-108e-4f9f-a106-cf2076c18902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:12.854624 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274de5b5-3c9f-463b-90fa-e502578be526 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:12.889457 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d729d3c-96e1-402a-9e7b-f9eeee59f693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:12.890604 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d729d3c-96e1-402a-9e7b-f9eeee59f693 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:12.890894 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d729d3c-96e1-402a-9e7b-f9eeee59f693 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:12.891183 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d729d3c-96e1-402a-9e7b-f9eeee59f693 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:12.899475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d729d3c-96e1-402a-9e7b-f9eeee59f693 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:12.900961 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d729d3c-96e1-402a-9e7b-f9eeee59f693 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:12.913279 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 410/807] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:12.914624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7760c4-108e-4f9f-a106-cf2076c18902 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:12.915473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-3d729d3c-96e1-402a-9e7b-f9eeee59f693 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 13:56:12.915473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:12.915473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:12.915473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:12.915473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:12.915473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:12.915473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:12.915473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:12.915473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:12.915473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:12.915473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:12.915473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:12.915473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:56:12.915473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:56:12.915473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:56:12.915473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:56:12.916450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:12.916450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:12.916450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:56:12.916450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:12.916450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:56:12.916450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:12.916450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:12.916450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:12.916450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:12.916450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:12.916450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:12.916450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:12.916450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:12.916450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:12.916450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:12.916450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:12.916450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:12.916450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:12.917403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:12.917403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:12.917403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:56:12.917403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:56:12.917403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:56:12.917403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:56:12.917403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:56:12.917403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:56:12.917403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 13:56:12.917403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:12.921218 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 398/808] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:56:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:12.921958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d729d3c-96e1-402a-9e7b-f9eeee59f693 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:12.961739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:13.018642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7760c4-108e-4f9f-a106-cf2076c18902 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:13.019227 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:13.026258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d729d3c-96e1-402a-9e7b-f9eeee59f693 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:13.054571 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd3c28-c1c5-46db-a74e-b8aba37233a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:13.055236 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd3c28-c1c5-46db-a74e-b8aba37233a2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:13.055524 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd3c28-c1c5-46db-a74e-b8aba37233a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:13.055804 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd3c28-c1c5-46db-a74e-b8aba37233a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:13.056551 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd3c28-c1c5-46db-a74e-b8aba37233a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:13.056859 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd3c28-c1c5-46db-a74e-b8aba37233a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:13.058239 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd3c28-c1c5-46db-a74e-b8aba37233a2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:13.059241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972e44e-f7f0-4252-ac5b-aafd2eda9f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:13.059886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972e44e-f7f0-4252-ac5b-aafd2eda9f27 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:13.060198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972e44e-f7f0-4252-ac5b-aafd2eda9f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:13.060477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972e44e-f7f0-4252-ac5b-aafd2eda9f27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:13.069397 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 411/809] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:13.070639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd3c28-c1c5-46db-a74e-b8aba37233a2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:13.134153 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972e44e-f7f0-4252-ac5b-aafd2eda9f27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:13.135655 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972e44e-f7f0-4252-ac5b-aafd2eda9f27 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:13.229808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:13.237723 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 399/810] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:56:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 13:56:13.238530 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972e44e-f7f0-4252-ac5b-aafd2eda9f27 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:13.296085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:13.299077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd3c28-c1c5-46db-a74e-b8aba37233a2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:13.332692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271c85e-5d3e-40a6-bb2e-222ec36e3f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:13.333492 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271c85e-5d3e-40a6-bb2e-222ec36e3f74 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:13.333767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271c85e-5d3e-40a6-bb2e-222ec36e3f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:13.334044 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271c85e-5d3e-40a6-bb2e-222ec36e3f74 None admin] PATH_INFO: `/v3/services/network` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:13.334696 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271c85e-5d3e-40a6-bb2e-222ec36e3f74 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:13.334904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972e44e-f7f0-4252-ac5b-aafd2eda9f27 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:13.336116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271c85e-5d3e-40a6-bb2e-222ec36e3f74 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:13.347616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-d271c85e-5d3e-40a6-bb2e-222ec36e3f74 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 13:56:13.347616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:13.347616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:56:13.347616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:56:13.347616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:56:13.347616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:56:13.347616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:56:13.347616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:13.347616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 13:56:13.347616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:13.347616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:13.347616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:13.347616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:13.347616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:13.347616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:13.347616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:13.347616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:13.348722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:13.348722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:13.348722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:13.348722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:13.348722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:13.348722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:13.348722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:56:13.348722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:56:13.348722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:56:13.348722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:56:13.348722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:13.348722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:13.348722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:13.348722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:13.348722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:13.348722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:13.349739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:13.349739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:13.349739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:13.349739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:13.349739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:13.349739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:13.349739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:13.349739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:13.349739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:13.349739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:13.349739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:56:13.349739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:13.349739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 13:56:13.349739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:13.349739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 412/811] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:56:13 2022] GET /identity/v3/services/network => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:13.351283 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271c85e-5d3e-40a6-bb2e-222ec36e3f74 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:13.372367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527cd48-ea89-4f98-a2d3-bd842a23d9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:13.373035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527cd48-ea89-4f98-a2d3-bd842a23d9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:13.373327 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527cd48-ea89-4f98-a2d3-bd842a23d9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:13.373609 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527cd48-ea89-4f98-a2d3-bd842a23d9f6 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:13.374449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527cd48-ea89-4f98-a2d3-bd842a23d9f6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:13.374766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527cd48-ea89-4f98-a2d3-bd842a23d9f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:13.376215 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527cd48-ea89-4f98-a2d3-bd842a23d9f6 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:13.392941 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 400/812] 10.222.0.94 () {56 vars in 1135 bytes} [Mon Jul 18 13:56:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:13.393725 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527cd48-ea89-4f98-a2d3-bd842a23d9f6 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:13.447865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:13.450384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271c85e-5d3e-40a6-bb2e-222ec36e3f74 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:13.484391 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbab4d7-30ed-45df-9c1f-2593b6174df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:13.485124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbab4d7-30ed-45df-9c1f-2593b6174df4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:13.485404 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbab4d7-30ed-45df-9c1f-2593b6174df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:13.485680 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbab4d7-30ed-45df-9c1f-2593b6174df4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:13.486328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbab4d7-30ed-45df-9c1f-2593b6174df4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:13.487691 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbab4d7-30ed-45df-9c1f-2593b6174df4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:13.494822 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527cd48-ea89-4f98-a2d3-bd842a23d9f6 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:13.499970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5bbab4d7-30ed-45df-9c1f-2593b6174df4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 13:56:13.499970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:13.499970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:56:13.499970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:56:13.499970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:56:13.499970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:56:13.499970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:56:13.499970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:13.499970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 13:56:13.499970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:13.499970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:13.499970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:13.499970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:13.499970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:13.499970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:13.499970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:13.501086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:13.501086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:13.501086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:13.501086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:13.501086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:13.501086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:13.501086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:13.501086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:56:13.501086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:56:13.501086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:56:13.501086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:56:13.501086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:13.501086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:13.501086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:13.501086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:13.501086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:13.502065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:13.502065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:13.502065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:13.502065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:13.502065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:13.502065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:13.502065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:13.502065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:13.502065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:13.502065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:13.502065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:13.502065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:56:13.502065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:13.502065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 13:56:13.502065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:13.504236 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 413/813] 10.222.0.94 () {56 vars in 1129 bytes} [Mon Jul 18 13:56:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:13.504929 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbab4d7-30ed-45df-9c1f-2593b6174df4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:13.530358 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7633b-138b-4491-87fd-5ac750ad098c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:13.531159 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7633b-138b-4491-87fd-5ac750ad098c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:13.531450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7633b-138b-4491-87fd-5ac750ad098c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:13.531734 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7633b-138b-4491-87fd-5ac750ad098c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:13.538245 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:13.538753 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff7633b-138b-4491-87fd-5ac750ad098c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:13.540349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff7633b-138b-4491-87fd-5ac750ad098c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:13.554339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-0ff7633b-138b-4491-87fd-5ac750ad098c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:13.554339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:13.554339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:13.554339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:13.554339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:13.554339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:13.554339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:13.554339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:13.554339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:13.554339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:13.554339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:13.554339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:13.554339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:56:13.554339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:56:13.554339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:56:13.555231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:56:13.555231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:13.555231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:13.555231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:13.555231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:13.555231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:13.555231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:13.555231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:13.555231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:13.555231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:13.555231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:13.555231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:13.555231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:13.555231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:13.555231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:13.555231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:13.555231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:13.556838 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:56:13.556838 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:56:13.556838 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:56:13.556838 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:56:13.556838 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:56:13.556838 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:56:13.556838 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:13.556838 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:13.556838 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 401/814] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:56:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:13.556838 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7633b-138b-4491-87fd-5ac750ad098c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:13.614061 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbab4d7-30ed-45df-9c1f-2593b6174df4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:13.649097 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9223d-b1b5-4d8c-b72f-30c0c75e68aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:13.649881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9223d-b1b5-4d8c-b72f-30c0c75e68aa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:13.650162 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9223d-b1b5-4d8c-b72f-30c0c75e68aa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:13.650434 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9223d-b1b5-4d8c-b72f-30c0c75e68aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:13.656497 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7633b-138b-4491-87fd-5ac750ad098c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:13.660705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b9223d-b1b5-4d8c-b72f-30c0c75e68aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:13.662145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b9223d-b1b5-4d8c-b72f-30c0c75e68aa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:13.669853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:13.679969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e1b9223d-b1b5-4d8c-b72f-30c0c75e68aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:13.679969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:13.679969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:13.679969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:13.679969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:13.679969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:13.679969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:13.679969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:13.679969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:13.679969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:13.679969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:13.679969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:13.679969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:13.679969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:13.679969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:13.680940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:13.680940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:13.680940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:13.680940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:13.680940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:13.680940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:13.680940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:13.680940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:13.680940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:13.680940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:13.680940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:13.680940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:13.680940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:13.680940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:13.680940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:13.680940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:13.680940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:13.682377 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:13.682377 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:13.682377 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:13.682377 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:13.682377 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 414/815] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:13.682377 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9223d-b1b5-4d8c-b72f-30c0c75e68aa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:13.694251 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59592ea-fc97-4153-970a-ad48097970c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:13.694996 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59592ea-fc97-4153-970a-ad48097970c9 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:13.695285 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59592ea-fc97-4153-970a-ad48097970c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:13.695563 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59592ea-fc97-4153-970a-ad48097970c9 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:13.696304 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59592ea-fc97-4153-970a-ad48097970c9 None admin] RBAC: Adding query filter params (name=network) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:13.696612 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59592ea-fc97-4153-970a-ad48097970c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:13.697979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59592ea-fc97-4153-970a-ad48097970c9 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:13.706977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 402/816] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Jul 18 13:56:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:13.706977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59592ea-fc97-4153-970a-ad48097970c9 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:13.765240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:13.784378 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9223d-b1b5-4d8c-b72f-30c0c75e68aa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:13.802086 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59592ea-fc97-4153-970a-ad48097970c9 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:13.816864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09c1ff-6a1b-40c6-a0e1-18a86e783f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:13.817659 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09c1ff-6a1b-40c6-a0e1-18a86e783f70 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:13.817944 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09c1ff-6a1b-40c6-a0e1-18a86e783f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:13.818223 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09c1ff-6a1b-40c6-a0e1-18a86e783f70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:13.829448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b09c1ff-6a1b-40c6-a0e1-18a86e783f70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:13.831016 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b09c1ff-6a1b-40c6-a0e1-18a86e783f70 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:13.835037 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3e687-824b-44a1-880b-5c43114cac57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:13.835772 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3e687-824b-44a1-880b-5c43114cac57 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:13.836095 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3e687-824b-44a1-880b-5c43114cac57 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:13.836370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3e687-824b-44a1-880b-5c43114cac57 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:13.837066 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b3e687-824b-44a1-880b-5c43114cac57 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:13.837377 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b3e687-824b-44a1-880b-5c43114cac57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:13.838709 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b3e687-824b-44a1-880b-5c43114cac57 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:13.843058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 403/817] 10.222.0.94 () {56 vars in 1124 bytes} [Mon Jul 18 13:56:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:13.843814 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3e687-824b-44a1-880b-5c43114cac57 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:13.847990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5b09c1ff-6a1b-40c6-a0e1-18a86e783f70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:13.847990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:13.847990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:13.847990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:13.847990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:13.847990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:13.847990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:13.847990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:13.847990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:13.847990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:13.847990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:13.847990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:13.847990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:13.847990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:13.847990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:13.849048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:13.849048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:13.849048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:13.849048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:13.849048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:13.849048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:13.849048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:13.849048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:13.849048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:13.849048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:13.849048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:13.849048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:13.849048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:13.849048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:13.849048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:13.849048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:13.849048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:13.855228 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:13.855228 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:13.855228 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:13.855228 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:13.855228 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 415/818] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:13.855228 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09c1ff-6a1b-40c6-a0e1-18a86e783f70 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:13.888231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:13.945889 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3e687-824b-44a1-880b-5c43114cac57 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:13.949627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09c1ff-6a1b-40c6-a0e1-18a86e783f70 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:13.978457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:13.979537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefce66-9bf0-4c02-a50a-316849dadce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:13.980278 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefce66-9bf0-4c02-a50a-316849dadce7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:13.980590 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefce66-9bf0-4c02-a50a-316849dadce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:13.980876 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefce66-9bf0-4c02-a50a-316849dadce7 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:13.981702 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefce66-9bf0-4c02-a50a-316849dadce7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:13.981972 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefce66-9bf0-4c02-a50a-316849dadce7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:13.983395 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefce66-9bf0-4c02-a50a-316849dadce7 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:13.984140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469cf0da-ab69-4f6b-a11b-98f282a2f16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:13.984812 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cf0da-ab69-4f6b-a11b-98f282a2f16f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:13.985100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cf0da-ab69-4f6b-a11b-98f282a2f16f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:13.985389 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cf0da-ab69-4f6b-a11b-98f282a2f16f None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:13.986140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469cf0da-ab69-4f6b-a11b-98f282a2f16f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:13.986451 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469cf0da-ab69-4f6b-a11b-98f282a2f16f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:13.987807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469cf0da-ab69-4f6b-a11b-98f282a2f16f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:13.995362 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 404/819] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:56:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:13.996185 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefce66-9bf0-4c02-a50a-316849dadce7 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:14.000011 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 416/820] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:14.000800 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469cf0da-ab69-4f6b-a11b-98f282a2f16f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:14.096245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:14.097710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469cf0da-ab69-4f6b-a11b-98f282a2f16f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:14.098812 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefce66-9bf0-4c02-a50a-316849dadce7 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:14.130687 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b5f10-01fa-4324-a531-f545689906f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:14.131028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb447f4-9f9f-40dd-b6a8-28f137c8997c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:14.131348 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b5f10-01fa-4324-a531-f545689906f2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:14.131634 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b5f10-01fa-4324-a531-f545689906f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:14.131746 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb447f4-9f9f-40dd-b6a8-28f137c8997c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:14.131922 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b5f10-01fa-4324-a531-f545689906f2 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:14.132621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb447f4-9f9f-40dd-b6a8-28f137c8997c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:14.132826 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb447f4-9f9f-40dd-b6a8-28f137c8997c None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:14.133135 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b5f10-01fa-4324-a531-f545689906f2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:14.133451 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b5f10-01fa-4324-a531-f545689906f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:14.133547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb447f4-9f9f-40dd-b6a8-28f137c8997c None admin] RBAC: Adding query filter params (type=network) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:14.133838 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb447f4-9f9f-40dd-b6a8-28f137c8997c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:14.134813 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b5f10-01fa-4324-a531-f545689906f2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:14.135177 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb447f4-9f9f-40dd-b6a8-28f137c8997c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:14.139711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 417/821] 10.222.0.94 () {56 vars in 1124 bytes} [Mon Jul 18 13:56:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:14.144649 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 405/822] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Jul 18 13:56:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:14.144784 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b5f10-01fa-4324-a531-f545689906f2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:14.144879 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb447f4-9f9f-40dd-b6a8-28f137c8997c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:14.198866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:14.237126 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb447f4-9f9f-40dd-b6a8-28f137c8997c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:14.243759 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b5f10-01fa-4324-a531-f545689906f2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:14.270526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c0a299-80a4-4a7a-a674-9ebfed5af504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:14.271404 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0a299-80a4-4a7a-a674-9ebfed5af504 None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:14.271687 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0a299-80a4-4a7a-a674-9ebfed5af504 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:14.271993 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0a299-80a4-4a7a-a674-9ebfed5af504 None admin] PATH_INFO: `/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/94d824bffc2c4af299c537056bc21734/roles/5c4ba827ac854bc69cc26c3bcd27dc5c` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:14.277299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d54e7-d9b1-4596-a405-ea1bb914690e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:14.277952 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d54e7-d9b1-4596-a405-ea1bb914690e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:14.278238 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d54e7-d9b1-4596-a405-ea1bb914690e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:14.278513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d54e7-d9b1-4596-a405-ea1bb914690e None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:14.279239 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d54e7-d9b1-4596-a405-ea1bb914690e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:14.279545 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d54e7-d9b1-4596-a405-ea1bb914690e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:14.280933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d54e7-d9b1-4596-a405-ea1bb914690e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:14.286035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c0a299-80a4-4a7a-a674-9ebfed5af504 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec94946acf91486ba3f5b4fe834b8076, user_id=94d824bffc2c4af299c537056bc21734, role_id=5c4ba827ac854bc69cc26c3bcd27dc5c)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:14.287453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c0a299-80a4-4a7a-a674-9ebfed5af504 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:14.291849 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 418/823] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:14.293050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d54e7-d9b1-4596-a405-ea1bb914690e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:14.301834 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 406/824] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:56:14 2022] PUT /identity/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/94d824bffc2c4af299c537056bc21734/roles/5c4ba827ac854bc69cc26c3bcd27dc5c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:56:14.302612 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c0a299-80a4-4a7a-a674-9ebfed5af504 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:14.320673 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:14.418174 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d54e7-d9b1-4596-a405-ea1bb914690e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:14.421452 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c0a299-80a4-4a7a-a674-9ebfed5af504 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:14.438759 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:14.464288 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2627efa-ce75-4842-925d-1c32bac96b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:14.464494 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e16b1-8265-4d87-9b64-645d0d6e5bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:14.464744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e16b1-8265-4d87-9b64-645d0d6e5bdc None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:14.464870 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2627efa-ce75-4842-925d-1c32bac96b3e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:14.465029 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e16b1-8265-4d87-9b64-645d0d6e5bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:14.465124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2627efa-ce75-4842-925d-1c32bac96b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:14.465308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e16b1-8265-4d87-9b64-645d0d6e5bdc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:14.465406 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2627efa-ce75-4842-925d-1c32bac96b3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:14.466051 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e16b1-8265-4d87-9b64-645d0d6e5bdc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:14.467416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e16b1-8265-4d87-9b64-645d0d6e5bdc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:14.473134 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2627efa-ce75-4842-925d-1c32bac96b3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:14.474575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2627efa-ce75-4842-925d-1c32bac96b3e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:14.488987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 407/825] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Jul 18 13:56:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:56:14.489787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e16b1-8265-4d87-9b64-645d0d6e5bdc None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:14.492260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f2627efa-ce75-4842-925d-1c32bac96b3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:14.492260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:14.492260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:14.492260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:14.492260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:14.492260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:14.492260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:14.492260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:14.492260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:14.492260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:14.492260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:14.492260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:14.492260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:56:14.492260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:56:14.492260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:56:14.493172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:56:14.493172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:14.493172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:14.493172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:14.493172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:14.493172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:14.493172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:14.493172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:14.493172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:14.493172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:14.493172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:14.493172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:14.493172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:14.493172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:14.493172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:14.493172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:14.493172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:14.494090 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:56:14.494090 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:56:14.494090 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:56:14.494090 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:56:14.494090 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:56:14.494090 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:56:14.494090 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:14.494090 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:14.494090 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 419/826] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:56:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:14.494585 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2627efa-ce75-4842-925d-1c32bac96b3e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:14.600615 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:14.617377 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2627efa-ce75-4842-925d-1c32bac96b3e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:14.653960 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e16b1-8265-4d87-9b64-645d0d6e5bdc None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:14.656798 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a656dd4e-cbaa-4af2-88f3-55f92465c58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:14.657598 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656dd4e-cbaa-4af2-88f3-55f92465c58a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:14.657882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656dd4e-cbaa-4af2-88f3-55f92465c58a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:14.658157 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656dd4e-cbaa-4af2-88f3-55f92465c58a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:14.665613 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a656dd4e-cbaa-4af2-88f3-55f92465c58a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:14.667066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a656dd4e-cbaa-4af2-88f3-55f92465c58a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:14.681926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a656dd4e-cbaa-4af2-88f3-55f92465c58a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:14.681926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:14.681926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:14.681926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:14.681926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:14.681926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:14.681926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:14.681926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:14.681926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:14.681926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:14.681926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:14.681926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:14.681926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:56:14.681926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:56:14.681926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:56:14.682829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:56:14.682829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:14.682829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:14.682829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:14.682829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:14.682829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:14.682829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:14.682829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:14.682829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:14.682829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:14.682829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:14.682829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:14.682829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:14.682829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:14.682829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:14.682829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:14.682829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:14.683728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:56:14.683728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:56:14.683728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:56:14.683728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:56:14.683728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:56:14.683728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:56:14.683728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:14.683728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:14.683728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 420/827] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:56:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:14.684281 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a656dd4e-cbaa-4af2-88f3-55f92465c58a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:14.701932 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d478c7b-fa38-4a7c-973d-5c46bec03b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:14.702606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d478c7b-fa38-4a7c-973d-5c46bec03b47 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:14.702895 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d478c7b-fa38-4a7c-973d-5c46bec03b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:14.703284 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d478c7b-fa38-4a7c-973d-5c46bec03b47 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:14.704784 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d478c7b-fa38-4a7c-973d-5c46bec03b47 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:14.706175 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d478c7b-fa38-4a7c-973d-5c46bec03b47 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:14.731271 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:14.732169 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 408/828] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Jul 18 13:56:14 2022] POST /identity/v3/endpoints => generated 327 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:56:14.733086 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d478c7b-fa38-4a7c-973d-5c46bec03b47 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:14.815993 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a656dd4e-cbaa-4af2-88f3-55f92465c58a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:14.873911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ed4ed-238f-48f3-b8c3-e2502c38d812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:14.874716 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ed4ed-238f-48f3-b8c3-e2502c38d812 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:14.875152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ed4ed-238f-48f3-b8c3-e2502c38d812 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:14.875373 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ed4ed-238f-48f3-b8c3-e2502c38d812 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:14.876211 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ed4ed-238f-48f3-b8c3-e2502c38d812 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:14.876585 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ed4ed-238f-48f3-b8c3-e2502c38d812 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:14.878227 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ed4ed-238f-48f3-b8c3-e2502c38d812 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:14.905709 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 421/829] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:56:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:14.921047 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ed4ed-238f-48f3-b8c3-e2502c38d812 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:14.922966 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d478c7b-fa38-4a7c-973d-5c46bec03b47 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:14.959576 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:14.975260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb8465-80e5-43f5-b671-66b3252850a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:14.976994 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb8465-80e5-43f5-b671-66b3252850a8 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:14.977515 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb8465-80e5-43f5-b671-66b3252850a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:14.978009 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb8465-80e5-43f5-b671-66b3252850a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:14.979003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb8465-80e5-43f5-b671-66b3252850a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:14.979527 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb8465-80e5-43f5-b671-66b3252850a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:14.981198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb8465-80e5-43f5-b671-66b3252850a8 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:14.994534 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 409/830] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:56:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:15.002976 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb8465-80e5-43f5-b671-66b3252850a8 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:15.038991 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:15.044709 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ed4ed-238f-48f3-b8c3-e2502c38d812 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:15.088302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb319e65-9426-4505-bfd3-6d57e85538ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:15.088302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb319e65-9426-4505-bfd3-6d57e85538ff None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:15.088302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb319e65-9426-4505-bfd3-6d57e85538ff None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:15.088660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb319e65-9426-4505-bfd3-6d57e85538ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:15.089395 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb319e65-9426-4505-bfd3-6d57e85538ff None admin] RBAC: Adding query filter params (role.id=c60a99de086f45c6820f7872a6b43feb, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=ac717fcd9fc1499db12cf852d45572a8) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:15.089716 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb319e65-9426-4505-bfd3-6d57e85538ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:15.091241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb319e65-9426-4505-bfd3-6d57e85538ff None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:15.099797 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 422/831] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:56:14 2022] GET /identity/v3/role_assignments?user.id=ac717fcd9fc1499db12cf852d45572a8&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=c60a99de086f45c6820f7872a6b43feb => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:15.130014 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb8465-80e5-43f5-b671-66b3252850a8 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:15.170022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757152f6-487b-4110-8634-3c02c592bbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:15.170942 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757152f6-487b-4110-8634-3c02c592bbd9 None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:15.171233 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757152f6-487b-4110-8634-3c02c592bbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:15.171517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757152f6-487b-4110-8634-3c02c592bbd9 None admin] PATH_INFO: `/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/cbf513ee25374e49b3d8ce28843e393a/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:15.173678 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757152f6-487b-4110-8634-3c02c592bbd9 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec94946acf91486ba3f5b4fe834b8076, user_id=cbf513ee25374e49b3d8ce28843e393a, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:15.175156 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757152f6-487b-4110-8634-3c02c592bbd9 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:15.193145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 410/832] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:56:15 2022] PUT /identity/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/cbf513ee25374e49b3d8ce28843e393a/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:56:16.587690 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e14fa-23b9-4ce1-ac29-5a0507ab75bd None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:16.588243 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e14fa-23b9-4ce1-ac29-5a0507ab75bd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:16.588243 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e14fa-23b9-4ce1-ac29-5a0507ab75bd None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:16.589833 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 423/833] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:16.595061 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e395b-b417-49f1-895d-592a744de6fa None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:16.595446 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e395b-b417-49f1-895d-592a744de6fa None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:16.595631 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e395b-b417-49f1-895d-592a744de6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:16.612250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-421e395b-b417-49f1-895d-592a744de6fa None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:16.613433 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-421e395b-b417-49f1-895d-592a744de6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:16.656202 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:16.669830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb4962-8c74-4769-ab97-56c76a938940 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:16.669830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb4962-8c74-4769-ab97-56c76a938940 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:16.669830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb4962-8c74-4769-ab97-56c76a938940 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:16.670482 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 424/834] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:16.675530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce31a5-1139-4722-9f94-825328391e7a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:16.676089 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce31a5-1139-4722-9f94-825328391e7a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:16.676599 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce31a5-1139-4722-9f94-825328391e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:16.689295 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 411/835] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:16.694393 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b6ce31a5-1139-4722-9f94-825328391e7a None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:16.695699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b6ce31a5-1139-4722-9f94-825328391e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:16.715034 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e395b-b417-49f1-895d-592a744de6fa None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:16.780073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 425/836] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 13:56:16.802558 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce31a5-1139-4722-9f94-825328391e7a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:16.825784 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e395b-b417-49f1-895d-592a744de6fa None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:16.865193 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cbcd9a-512b-4745-9882-3bf7e1d3597a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:16.866434 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbcd9a-512b-4745-9882-3bf7e1d3597a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:16.866938 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbcd9a-512b-4745-9882-3bf7e1d3597a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:16.867417 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cbcd9a-512b-4745-9882-3bf7e1d3597a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:16.878371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cbcd9a-512b-4745-9882-3bf7e1d3597a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:56:16.887135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-b9cbcd9a-512b-4745-9882-3bf7e1d3597a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:56:16.887135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:16.887135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:16.887135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:16.887135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:16.887135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:16.887135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:16.887135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:16.887135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:16.887135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:16.887135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:16.887135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:16.887135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:56:16.887135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:56:16.887135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:56:16.887135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:56:16.888107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:56:16.888107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:56:16.888107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:56:16.888107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:56:16.888107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:16.888107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:16.888107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:16.888107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:16.888107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:16.888107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:16.888107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:16.888107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:16.888107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:16.888107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:16.888107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:16.888107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:16.888965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:16.888965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:16.888965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:16.888965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:16.888965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:56:16.888965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:56:16.888965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:56:16.888965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:56:16.888965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:56:16.888965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:56:16.888965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:56:16.888965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:16.891725 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 412/837] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:56:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:16.898324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cbcd9a-512b-4745-9882-3bf7e1d3597a None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:16.906554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:16.924844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce31a5-1139-4722-9f94-825328391e7a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:16.942604 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:16.974568 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdad125-c0ec-4598-bf23-18cf00dbe5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:16.976333 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdad125-c0ec-4598-bf23-18cf00dbe5ca None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:16.976846 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdad125-c0ec-4598-bf23-18cf00dbe5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:16.977321 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdad125-c0ec-4598-bf23-18cf00dbe5ca None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:16.978171 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdad125-c0ec-4598-bf23-18cf00dbe5ca None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:16.979761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdad125-c0ec-4598-bf23-18cf00dbe5ca None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:16.992673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-1fdad125-c0ec-4598-bf23-18cf00dbe5ca None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 13:56:16.992673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:16.992673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:56:16.992673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:56:16.992673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:56:16.992673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:56:16.992673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:56:16.992673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:16.992673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 13:56:16.992673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:16.992673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:16.992673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:16.992673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:16.992673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:16.992673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:16.992673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:16.992673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:16.993692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:16.993692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:16.993692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:16.993692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:16.993692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:16.993692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:16.993692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:56:16.993692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:56:16.993692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:56:16.993692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:56:16.993692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:16.993692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:16.993692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:16.993692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:16.993692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:16.993692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:16.994548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:16.994548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:16.994548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:16.994548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:16.994548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:16.994548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:16.994548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:16.994548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:16.994548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:16.994548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:16.994548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:56:16.994548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:16.994548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 13:56:16.994548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:17.001948 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 426/838] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Jul 18 13:56:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:17.003028 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdad125-c0ec-4598-bf23-18cf00dbe5ca None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:17.026744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cbcd9a-512b-4745-9882-3bf7e1d3597a None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:17.065414 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c705fde-8691-499e-9780-1e705d0e8916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:17.066458 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c705fde-8691-499e-9780-1e705d0e8916 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:17.066947 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c705fde-8691-499e-9780-1e705d0e8916 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:17.067424 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c705fde-8691-499e-9780-1e705d0e8916 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:17.068436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c705fde-8691-499e-9780-1e705d0e8916 None admin] RBAC: Adding query filter params (name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:17.068950 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c705fde-8691-499e-9780-1e705d0e8916 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:17.071167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c705fde-8691-499e-9780-1e705d0e8916 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:17.080402 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 413/839] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Jul 18 13:56:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:17.085747 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c705fde-8691-499e-9780-1e705d0e8916 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:17.134754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdad125-c0ec-4598-bf23-18cf00dbe5ca None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:17.177158 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7b9f3-e9f4-4f01-9ccf-a461a6d74aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:17.177846 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7b9f3-e9f4-4f01-9ccf-a461a6d74aac None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:17.178133 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7b9f3-e9f4-4f01-9ccf-a461a6d74aac None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:17.178410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7b9f3-e9f4-4f01-9ccf-a461a6d74aac None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:17.179673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7b9f3-e9f4-4f01-9ccf-a461a6d74aac None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:17.180001 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7b9f3-e9f4-4f01-9ccf-a461a6d74aac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:17.181396 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7b9f3-e9f4-4f01-9ccf-a461a6d74aac None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:17.187987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 427/840] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Jul 18 13:56:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:17.192927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7b9f3-e9f4-4f01-9ccf-a461a6d74aac None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:17.215570 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:17.309723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7b9f3-e9f4-4f01-9ccf-a461a6d74aac None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:17.316952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:17.325170 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c705fde-8691-499e-9780-1e705d0e8916 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:17.345646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca127cfb-37ff-4861-b702-928746c6c07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:17.346306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca127cfb-37ff-4861-b702-928746c6c07a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:17.346596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca127cfb-37ff-4861-b702-928746c6c07a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:17.346872 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca127cfb-37ff-4861-b702-928746c6c07a None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:17.347578 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca127cfb-37ff-4861-b702-928746c6c07a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:17.347886 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca127cfb-37ff-4861-b702-928746c6c07a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:17.349278 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca127cfb-37ff-4861-b702-928746c6c07a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:17.353725 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 428/841] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Jul 18 13:56:17 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:17.354902 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99303a6e-a727-4658-9df2-1676457af62f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:17.355189 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99303a6e-a727-4658-9df2-1676457af62f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:17.355473 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99303a6e-a727-4658-9df2-1676457af62f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:17.356987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 429/842] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:17.362259 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4ed1b-3cdd-41c0-87da-8466a32017f3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:17.362550 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4ed1b-3cdd-41c0-87da-8466a32017f3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:17.362834 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4ed1b-3cdd-41c0-87da-8466a32017f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:17.382864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3bd27-dee7-4e78-8b57-cbc57e47c602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:17.384167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3bd27-dee7-4e78-8b57-cbc57e47c602 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:17.384736 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3bd27-dee7-4e78-8b57-cbc57e47c602 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:17.385286 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3bd27-dee7-4e78-8b57-cbc57e47c602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:17.388533 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-90d4ed1b-3cdd-41c0-87da-8466a32017f3 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:17.389773 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-90d4ed1b-3cdd-41c0-87da-8466a32017f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:17.399421 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3bd27-dee7-4e78-8b57-cbc57e47c602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:17.401560 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3bd27-dee7-4e78-8b57-cbc57e47c602 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:17.423279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-9ce3bd27-dee7-4e78-8b57-cbc57e47c602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:17.423279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:17.423279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:17.423279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:17.423279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:17.423279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:17.423279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:17.423279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:17.423279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:17.423279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:17.423279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:17.423279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:17.423279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:17.423279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:17.423279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:17.424383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:17.424383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:17.424383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:17.424383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:17.424383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:17.424383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:17.424383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:17.424383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:17.424383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:17.424383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:17.424383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:17.424383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:17.424383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:17.424383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:17.424383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:17.424383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:17.424383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:17.425294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:17.425294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:17.425294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:17.425294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:17.425294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 414/843] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:17.426362 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2a2b9-4e64-4515-a19f-454a356bc0ed None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:17.426655 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2a2b9-4e64-4515-a19f-454a356bc0ed None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:17.426994 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2a2b9-4e64-4515-a19f-454a356bc0ed None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:17.428391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 415/844] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:17.430436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2a2b9-4e64-4515-a19f-454a356bc0ed None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:17.470095 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 430/845] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 13:56:17.471909 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d5bd7-c001-4913-889b-b85ff1de6465 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:17.472245 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d5bd7-c001-4913-889b-b85ff1de6465 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:17.472529 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d5bd7-c001-4913-889b-b85ff1de6465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:17.496430 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-045d5bd7-c001-4913-889b-b85ff1de6465 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:17.497925 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-045d5bd7-c001-4913-889b-b85ff1de6465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:17.500236 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:17.561436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2a2b9-4e64-4515-a19f-454a356bc0ed None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:17.578773 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 431/846] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:17.579853 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d5bd7-c001-4913-889b-b85ff1de6465 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:17.609760 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f151a7e-f838-46df-995a-3da883b71639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:17.611729 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f151a7e-f838-46df-995a-3da883b71639 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:17.612246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f151a7e-f838-46df-995a-3da883b71639 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:17.612869 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f151a7e-f838-46df-995a-3da883b71639 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:17.614374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f151a7e-f838-46df-995a-3da883b71639 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:17.614908 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f151a7e-f838-46df-995a-3da883b71639 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:17.616721 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f151a7e-f838-46df-995a-3da883b71639 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:17.633018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 416/847] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:17.634426 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f151a7e-f838-46df-995a-3da883b71639 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:17.649152 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:17.685320 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d5bd7-c001-4913-889b-b85ff1de6465 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:17.722426 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459de62f-5c3e-48ed-aad9-8ed5326032b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:17.723224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459de62f-5c3e-48ed-aad9-8ed5326032b6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:17.723516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459de62f-5c3e-48ed-aad9-8ed5326032b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:17.723920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459de62f-5c3e-48ed-aad9-8ed5326032b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:17.731259 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:17.736341 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459de62f-5c3e-48ed-aad9-8ed5326032b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:56:17.745754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-459de62f-5c3e-48ed-aad9-8ed5326032b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:56:17.745754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:17.745754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:17.745754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:17.745754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:17.745754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:17.745754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:17.745754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:17.745754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:17.745754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:17.745754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:17.745754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:17.745754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:56:17.745754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:56:17.745754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:56:17.745754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:56:17.746698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:56:17.746698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:56:17.746698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:56:17.746698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:56:17.746698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:17.746698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:17.746698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:17.746698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:17.746698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:17.746698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:17.746698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:17.746698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:17.746698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:17.746698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:17.746698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:17.746698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:17.752660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:17.752660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:17.752660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:17.752660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:17.752660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:56:17.752660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:56:17.752660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:56:17.752660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:56:17.752660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:56:17.752660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:56:17.752660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:56:17.752660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:17.752660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 432/848] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:56:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:17.753359 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f151a7e-f838-46df-995a-3da883b71639 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:17.753436 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459de62f-5c3e-48ed-aad9-8ed5326032b6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:17.791323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d6436-3f11-4a04-9749-6e8d0b0e85d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:17.792687 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d6436-3f11-4a04-9749-6e8d0b0e85d4 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:17.792980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d6436-3f11-4a04-9749-6e8d0b0e85d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:17.793261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d6436-3f11-4a04-9749-6e8d0b0e85d4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:17.794347 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d6436-3f11-4a04-9749-6e8d0b0e85d4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:17.796349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d6436-3f11-4a04-9749-6e8d0b0e85d4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:17.812415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-f48d6436-3f11-4a04-9749-6e8d0b0e85d4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 13:56:17.812415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:17.812415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:56:17.812415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:56:17.812415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:56:17.812415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:56:17.812415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:56:17.812415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:17.812415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 13:56:17.812415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:17.812415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:17.812415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:17.812415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:17.812415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:17.812415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:17.812415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:17.813580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:17.813580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:17.813580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:17.813580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:17.813580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:17.813580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:17.813580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:17.813580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:56:17.813580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:56:17.813580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:56:17.813580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:56:17.813580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:17.813580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:17.813580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:17.813580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:17.813580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:17.814449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:17.814449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:17.814449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:17.814449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:17.814449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:17.814449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:17.814449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:17.814449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:17.814449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:17.814449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:17.814449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:17.814449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:56:17.814449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:17.814449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 13:56:17.814449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:17.814449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 417/849] 10.222.0.94 () {56 vars in 1132 bytes} [Mon Jul 18 13:56:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:17.815314 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d6436-3f11-4a04-9749-6e8d0b0e85d4 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:17.868414 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459de62f-5c3e-48ed-aad9-8ed5326032b6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:17.894051 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:17.906369 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11ecb8-5724-4282-a5ce-25058033b150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:17.907592 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11ecb8-5724-4282-a5ce-25058033b150 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:17.908255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11ecb8-5724-4282-a5ce-25058033b150 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:17.908639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11ecb8-5724-4282-a5ce-25058033b150 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:17.918071 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d11ecb8-5724-4282-a5ce-25058033b150 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:17.919734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d11ecb8-5724-4282-a5ce-25058033b150 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:17.926443 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d6436-3f11-4a04-9749-6e8d0b0e85d4 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:17.935019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-8d11ecb8-5724-4282-a5ce-25058033b150 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 13:56:17.935019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:17.935019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:17.935019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:17.935019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:17.935019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:17.935019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:17.935019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:17.935019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:17.935019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:17.935019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:17.935019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:17.935019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:56:17.935019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:56:17.935019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:56:17.935019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:56:17.936145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:17.936145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:17.936145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:56:17.936145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:17.936145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:56:17.936145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:17.936145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:17.936145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:17.936145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:17.936145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:17.936145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:17.936145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:17.936145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:17.936145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:17.936145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:17.936145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:17.936145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:17.936145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:17.937149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:17.937149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:17.937149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:56:17.937149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:56:17.937149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:56:17.937149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:56:17.937149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:56:17.937149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:56:17.937149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 13:56:17.937149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:17.941227 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 433/850] 10.222.0.94 () {56 vars in 1117 bytes} [Mon Jul 18 13:56:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:17.942028 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11ecb8-5724-4282-a5ce-25058033b150 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:17.972384 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb17c26-cd9b-4ef5-b276-bc6aa1f0042a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:17.973059 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb17c26-cd9b-4ef5-b276-bc6aa1f0042a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:17.973347 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb17c26-cd9b-4ef5-b276-bc6aa1f0042a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:17.973667 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb17c26-cd9b-4ef5-b276-bc6aa1f0042a None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:17.974421 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb17c26-cd9b-4ef5-b276-bc6aa1f0042a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:17.974731 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb17c26-cd9b-4ef5-b276-bc6aa1f0042a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:17.976736 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb17c26-cd9b-4ef5-b276-bc6aa1f0042a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:17.989697 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 418/851] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:56:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:17.989697 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb17c26-cd9b-4ef5-b276-bc6aa1f0042a None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:18.007341 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:18.052219 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11ecb8-5724-4282-a5ce-25058033b150 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:18.087783 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6277e1-e227-4faf-99a9-58583a628d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:18.088477 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6277e1-e227-4faf-99a9-58583a628d89 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:18.088767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6277e1-e227-4faf-99a9-58583a628d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:18.089047 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6277e1-e227-4faf-99a9-58583a628d89 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:18.089758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6277e1-e227-4faf-99a9-58583a628d89 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:18.090071 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6277e1-e227-4faf-99a9-58583a628d89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:18.091443 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6277e1-e227-4faf-99a9-58583a628d89 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:18.096001 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 434/852] 10.222.0.94 () {56 vars in 1126 bytes} [Mon Jul 18 13:56:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:18.096783 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6277e1-e227-4faf-99a9-58583a628d89 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:18.103028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb17c26-cd9b-4ef5-b276-bc6aa1f0042a None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:18.135017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:18.141190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e481ac-0b02-4f17-a4dc-f2099eb21dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:18.142067 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e481ac-0b02-4f17-a4dc-f2099eb21dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:18.142552 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e481ac-0b02-4f17-a4dc-f2099eb21dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:18.143007 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e481ac-0b02-4f17-a4dc-f2099eb21dd4 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:18.144054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e481ac-0b02-4f17-a4dc-f2099eb21dd4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:18.144543 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e481ac-0b02-4f17-a4dc-f2099eb21dd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:18.146109 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e481ac-0b02-4f17-a4dc-f2099eb21dd4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:18.164995 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 419/853] 10.222.0.94 () {56 vars in 1149 bytes} [Mon Jul 18 13:56:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:18.165977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e481ac-0b02-4f17-a4dc-f2099eb21dd4 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:18.211622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6277e1-e227-4faf-99a9-58583a628d89 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:18.235719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:18.250158 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c7313-829b-4d5c-addc-a0d516817e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:18.251210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c7313-829b-4d5c-addc-a0d516817e31 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:18.251641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c7313-829b-4d5c-addc-a0d516817e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:18.252129 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c7313-829b-4d5c-addc-a0d516817e31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:18.263385 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c7313-829b-4d5c-addc-a0d516817e31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:18.265005 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c7313-829b-4d5c-addc-a0d516817e31 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:18.274743 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e481ac-0b02-4f17-a4dc-f2099eb21dd4 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:18.282632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-906c7313-829b-4d5c-addc-a0d516817e31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:18.282632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:18.282632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:18.282632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:18.282632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:18.282632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:18.282632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:18.282632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:18.282632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:18.282632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:18.282632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:18.282632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:18.282632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:18.282632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:18.282632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:18.283688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:18.283688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:18.283688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:18.283688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:18.283688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:18.283688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:18.283688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:18.283688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:18.283688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:18.283688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:18.283688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:18.283688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:18.283688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:18.283688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:18.283688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:18.283688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:18.283688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:18.284645 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:18.284645 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:18.284645 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:18.284645 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:18.290609 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 435/854] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:18.290609 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c7313-829b-4d5c-addc-a0d516817e31 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:18.311378 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01c9d6-33ca-469c-a441-d4535160e328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:18.312075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01c9d6-33ca-469c-a441-d4535160e328 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:18.312616 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01c9d6-33ca-469c-a441-d4535160e328 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:18.312616 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01c9d6-33ca-469c-a441-d4535160e328 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:18.313344 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01c9d6-33ca-469c-a441-d4535160e328 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:18.313653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01c9d6-33ca-469c-a441-d4535160e328 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:18.315018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01c9d6-33ca-469c-a441-d4535160e328 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:18.320278 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 420/855] 10.222.0.94 () {56 vars in 1126 bytes} [Mon Jul 18 13:56:18 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:18.333046 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01c9d6-33ca-469c-a441-d4535160e328 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:18.404483 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:18.427706 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c7313-829b-4d5c-addc-a0d516817e31 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:18.458002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01c9d6-33ca-469c-a441-d4535160e328 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:18.476947 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da820e60-609f-4f51-a843-13d270a5b0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:18.478211 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da820e60-609f-4f51-a843-13d270a5b0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:18.479028 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da820e60-609f-4f51-a843-13d270a5b0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:18.479498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da820e60-609f-4f51-a843-13d270a5b0b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:18.506079 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c20af-eb53-4a3e-a46a-9544e9c4117c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:18.506883 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c20af-eb53-4a3e-a46a-9544e9c4117c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:18.507222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c20af-eb53-4a3e-a46a-9544e9c4117c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:18.507596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c20af-eb53-4a3e-a46a-9544e9c4117c None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:18.508973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6c20af-eb53-4a3e-a46a-9544e9c4117c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:18.508973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6c20af-eb53-4a3e-a46a-9544e9c4117c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:18.510408 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6c20af-eb53-4a3e-a46a-9544e9c4117c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:18.522857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 421/856] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:18.528363 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c20af-eb53-4a3e-a46a-9544e9c4117c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:18.531094 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:18.558575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da820e60-609f-4f51-a843-13d270a5b0b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:18.560065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da820e60-609f-4f51-a843-13d270a5b0b6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:18.661930 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c20af-eb53-4a3e-a46a-9544e9c4117c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:18.682375 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 436/857] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:56:18 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 13:56:18.688932 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da820e60-609f-4f51-a843-13d270a5b0b6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:18.706283 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:18.719163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8cd74e-dd22-4ba0-a222-c7254cb623c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:18.720649 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8cd74e-dd22-4ba0-a222-c7254cb623c5 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:18.721357 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8cd74e-dd22-4ba0-a222-c7254cb623c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:18.722238 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8cd74e-dd22-4ba0-a222-c7254cb623c5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:18.731394 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8cd74e-dd22-4ba0-a222-c7254cb623c5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:18.733625 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8cd74e-dd22-4ba0-a222-c7254cb623c5 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:18.754103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-7c8cd74e-dd22-4ba0-a222-c7254cb623c5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 13:56:18.754103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:18.754103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:18.754103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:18.754103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:18.754103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:18.754103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:18.754103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:18.754103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:18.754103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:18.754103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:18.754103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:18.754103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:56:18.754103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:56:18.754103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:56:18.754103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:56:18.755054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:18.755054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:18.755054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:56:18.755054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:18.755054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:56:18.755054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:18.755054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:18.755054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:18.755054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:18.755054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:18.755054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:18.755054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:18.755054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:18.755054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:18.755054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:18.755054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:18.755054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:18.755054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:18.756018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:18.756018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:18.756018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:56:18.756018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:56:18.756018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:56:18.756018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:56:18.756018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:56:18.756018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:56:18.756018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 13:56:18.756018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:18.762629 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 422/858] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:56:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:18.765124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8cd74e-dd22-4ba0-a222-c7254cb623c5 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:18.775442 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:18.803211 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da820e60-609f-4f51-a843-13d270a5b0b6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:18.853371 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52279df1-3767-46a8-9137-03d8bd9b7797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:18.854644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52279df1-3767-46a8-9137-03d8bd9b7797 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:18.855371 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52279df1-3767-46a8-9137-03d8bd9b7797 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:18.856807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52279df1-3767-46a8-9137-03d8bd9b7797 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:18.870849 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52279df1-3767-46a8-9137-03d8bd9b7797 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:18.873328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52279df1-3767-46a8-9137-03d8bd9b7797 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:18.888068 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8cd74e-dd22-4ba0-a222-c7254cb623c5 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:18.897971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-52279df1-3767-46a8-9137-03d8bd9b7797 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:18.897971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:18.897971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:18.897971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:18.897971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:18.897971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:18.897971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:18.897971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:18.897971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:18.897971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:18.897971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:18.897971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:18.897971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:18.897971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:18.897971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:18.898885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:18.898885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:18.898885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:18.898885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:18.898885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:18.898885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:18.898885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:18.898885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:18.898885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:18.898885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:18.898885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:18.898885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:18.898885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:18.898885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:18.898885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:18.898885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:18.898885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:18.899774 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:18.899774 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:18.899774 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:18.899774 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:18.906124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 437/859] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:18.907248 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52279df1-3767-46a8-9137-03d8bd9b7797 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:18.931206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8b26b-1446-4c52-99f6-88aa7a747dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:18.932137 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8b26b-1446-4c52-99f6-88aa7a747dbf None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:18.932562 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8b26b-1446-4c52-99f6-88aa7a747dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:18.932865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8b26b-1446-4c52-99f6-88aa7a747dbf None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:18.933811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b8b26b-1446-4c52-99f6-88aa7a747dbf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:18.934275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b8b26b-1446-4c52-99f6-88aa7a747dbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:18.935767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b8b26b-1446-4c52-99f6-88aa7a747dbf None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:18.958405 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 423/860] 10.222.0.94 () {56 vars in 1135 bytes} [Mon Jul 18 13:56:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:18.964978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8b26b-1446-4c52-99f6-88aa7a747dbf None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:18.997671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:19.041131 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52279df1-3767-46a8-9137-03d8bd9b7797 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:19.069829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:19.072272 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8b26b-1446-4c52-99f6-88aa7a747dbf None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:19.078933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2a95d-0b0c-4422-b3a4-51f2c46645e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:19.079855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2a95d-0b0c-4422-b3a4-51f2c46645e1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:19.080166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2a95d-0b0c-4422-b3a4-51f2c46645e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:19.080687 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2a95d-0b0c-4422-b3a4-51f2c46645e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:19.081663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2a95d-0b0c-4422-b3a4-51f2c46645e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:19.082168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2a95d-0b0c-4422-b3a4-51f2c46645e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:19.083659 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2a95d-0b0c-4422-b3a4-51f2c46645e1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:19.101571 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 438/861] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:19.102644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2a95d-0b0c-4422-b3a4-51f2c46645e1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:19.113048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513f1c9-e564-4c3b-a735-ce6357072c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:19.114059 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513f1c9-e564-4c3b-a735-ce6357072c64 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:19.114553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513f1c9-e564-4c3b-a735-ce6357072c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:19.115031 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513f1c9-e564-4c3b-a735-ce6357072c64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:19.184843 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9513f1c9-e564-4c3b-a735-ce6357072c64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:19.186639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9513f1c9-e564-4c3b-a735-ce6357072c64 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:19.212251 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2a95d-0b0c-4422-b3a4-51f2c46645e1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:19.218014 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:19.251489 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c76850-8637-4ca9-8018-65cca394e88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:19.252733 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c76850-8637-4ca9-8018-65cca394e88c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:19.253295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c76850-8637-4ca9-8018-65cca394e88c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:19.253714 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c76850-8637-4ca9-8018-65cca394e88c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:19.261325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c76850-8637-4ca9-8018-65cca394e88c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:19.262906 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c76850-8637-4ca9-8018-65cca394e88c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:19.278240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e2c76850-8637-4ca9-8018-65cca394e88c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:19.278240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:19.278240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:19.278240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:19.278240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:19.278240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:19.278240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:19.278240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:19.278240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:19.278240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:19.278240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:19.278240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:19.278240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:56:19.278240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:56:19.278240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:56:19.279329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:56:19.279329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:19.279329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:19.279329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:19.279329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:19.279329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:19.279329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:19.279329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:19.279329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:19.279329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:19.279329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:19.279329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:19.279329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:19.279329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:19.279329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:19.279329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:19.279329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:19.280467 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:56:19.280467 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:56:19.280467 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:56:19.280467 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:56:19.280467 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:56:19.280467 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:56:19.280467 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:19.280467 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:19.287219 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 439/862] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:56:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:19.289104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c76850-8637-4ca9-8018-65cca394e88c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:19.297934 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 424/863] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:56:18 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 13:56:19.304449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513f1c9-e564-4c3b-a735-ce6357072c64 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:19.343543 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:19.408626 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9513f1c9-e564-4c3b-a735-ce6357072c64 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:19.413611 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c76850-8637-4ca9-8018-65cca394e88c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:19.441279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:19.447552 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b5393-5b19-4e93-85f3-ee7c9e34cc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:19.448381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b5393-5b19-4e93-85f3-ee7c9e34cc06 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:19.448667 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b5393-5b19-4e93-85f3-ee7c9e34cc06 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:19.448957 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b5393-5b19-4e93-85f3-ee7c9e34cc06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:19.452470 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c613ee6-8c6e-4669-91bf-8c850ffb17ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:19.453134 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c613ee6-8c6e-4669-91bf-8c850ffb17ef None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:19.453419 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c613ee6-8c6e-4669-91bf-8c850ffb17ef None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:19.453695 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c613ee6-8c6e-4669-91bf-8c850ffb17ef None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:19.454747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c613ee6-8c6e-4669-91bf-8c850ffb17ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:19.455056 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c613ee6-8c6e-4669-91bf-8c850ffb17ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:19.456544 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c613ee6-8c6e-4669-91bf-8c850ffb17ef None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:19.461414 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b5393-5b19-4e93-85f3-ee7c9e34cc06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:19.463139 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b5393-5b19-4e93-85f3-ee7c9e34cc06 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:19.469728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 440/864] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:56:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:19.471085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902b094-b1a7-494d-81bb-80e7e7ee9c7b None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:19.471377 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902b094-b1a7-494d-81bb-80e7e7ee9c7b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:19.471653 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902b094-b1a7-494d-81bb-80e7e7ee9c7b None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:19.473133 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 441/865] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:19.478991 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210faec-4269-4d08-a2c0-d3494436b366 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:19.479278 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210faec-4269-4d08-a2c0-d3494436b366 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:19.479549 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210faec-4269-4d08-a2c0-d3494436b366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:19.488857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-a80b5393-5b19-4e93-85f3-ee7c9e34cc06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:19.488857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:19.488857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:19.488857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:19.488857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:19.488857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:19.488857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:19.488857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:19.488857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:19.488857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:19.488857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:19.488857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:19.488857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:19.488857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:19.488857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:19.489913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:19.489913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:19.489913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:19.489913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:19.489913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:19.489913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:19.489913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:19.489913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:19.489913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:19.489913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:19.489913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:19.489913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:19.489913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:19.489913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:19.489913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:19.489913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:19.489913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:19.491005 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:19.491005 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:19.491005 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:19.491005 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:19.497446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d210faec-4269-4d08-a2c0-d3494436b366 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:19.497714 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 425/866] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:19.498265 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d210faec-4269-4d08-a2c0-d3494436b366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:19.498840 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b5393-5b19-4e93-85f3-ee7c9e34cc06 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:19.565297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 442/867] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:19.566129 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210faec-4269-4d08-a2c0-d3494436b366 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:19.610904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b5393-5b19-4e93-85f3-ee7c9e34cc06 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:19.612173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:19.649984 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98e884-3c22-42cc-be84-ad57d04ec629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:19.650876 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98e884-3c22-42cc-be84-ad57d04ec629 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:19.651167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98e884-3c22-42cc-be84-ad57d04ec629 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:19.651449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98e884-3c22-42cc-be84-ad57d04ec629 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:19.652313 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98e884-3c22-42cc-be84-ad57d04ec629 None admin] RBAC: Adding query filter params (role.id=5c4ba827ac854bc69cc26c3bcd27dc5c, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=94d824bffc2c4af299c537056bc21734) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:19.652630 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98e884-3c22-42cc-be84-ad57d04ec629 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:19.654053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98e884-3c22-42cc-be84-ad57d04ec629 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:19.667971 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 426/868] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:56:19 2022] GET /identity/v3/role_assignments?user.id=94d824bffc2c4af299c537056bc21734&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=5c4ba827ac854bc69cc26c3bcd27dc5c => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:19.667971 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98e884-3c22-42cc-be84-ad57d04ec629 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:19.698522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210faec-4269-4d08-a2c0-d3494436b366 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:19.728502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:19.755915 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b68ae-d948-4312-bfa8-98494c59f839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:19.756636 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b68ae-d948-4312-bfa8-98494c59f839 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:19.757478 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b68ae-d948-4312-bfa8-98494c59f839 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:19.757763 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b68ae-d948-4312-bfa8-98494c59f839 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:19.758511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b68ae-d948-4312-bfa8-98494c59f839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:19.758825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b68ae-d948-4312-bfa8-98494c59f839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:19.760329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b68ae-d948-4312-bfa8-98494c59f839 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:19.777726 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 443/869] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:19.783441 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b68ae-d948-4312-bfa8-98494c59f839 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:19.793467 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98e884-3c22-42cc-be84-ad57d04ec629 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:19.830665 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653420bf-1e87-4cc5-b0cc-7953fda693db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:19.831337 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653420bf-1e87-4cc5-b0cc-7953fda693db None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:19.831626 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653420bf-1e87-4cc5-b0cc-7953fda693db None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:19.831908 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653420bf-1e87-4cc5-b0cc-7953fda693db None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:19.832970 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653420bf-1e87-4cc5-b0cc-7953fda693db None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:19.834359 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653420bf-1e87-4cc5-b0cc-7953fda693db None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:19.856030 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 427/870] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Jul 18 13:56:19 2022] POST /identity/v3/services => generated 251 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:56:19.900680 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:19.908640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b68ae-d948-4312-bfa8-98494c59f839 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:19.963199 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b59849-a4f2-4531-b982-e1b783f83677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:19.963934 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b59849-a4f2-4531-b982-e1b783f83677 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:19.964251 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b59849-a4f2-4531-b982-e1b783f83677 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:19.964530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b59849-a4f2-4531-b982-e1b783f83677 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:19.972385 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b59849-a4f2-4531-b982-e1b783f83677 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:19.973878 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b59849-a4f2-4531-b982-e1b783f83677 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:19.990128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-56b59849-a4f2-4531-b982-e1b783f83677 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:19.990128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:19.990128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:19.990128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:19.990128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:19.990128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:19.990128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:19.990128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:19.990128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:19.990128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:19.990128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:19.990128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:19.990128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:56:19.990128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:56:19.990128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:56:19.991185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:56:19.991185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:19.991185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:19.991185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:19.991185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:19.991185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:19.991185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:19.991185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:19.991185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:19.991185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:19.991185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:19.991185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:19.991185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:19.991185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:19.991185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:19.991185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:19.991185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:19.992630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:56:19.992630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:56:19.992630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:56:19.992630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:56:19.992630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:56:19.992630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:56:19.992630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:19.992630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:19.992630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 444/871] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:56:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:19.997996 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653420bf-1e87-4cc5-b0cc-7953fda693db None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:20.109775 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653420bf-1e87-4cc5-b0cc-7953fda693db None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:20.119754 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:20.146548 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ba07c-829a-468a-9279-6c54eaff2b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:20.147216 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ba07c-829a-468a-9279-6c54eaff2b10 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:20.148118 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ba07c-829a-468a-9279-6c54eaff2b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:20.148402 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ba07c-829a-468a-9279-6c54eaff2b10 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:20.149193 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ba07c-829a-468a-9279-6c54eaff2b10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:20.149498 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ba07c-829a-468a-9279-6c54eaff2b10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:20.150916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ba07c-829a-468a-9279-6c54eaff2b10 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:20.163818 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 428/872] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:56:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:20.173008 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b59849-a4f2-4531-b982-e1b783f83677 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:20.282993 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b59849-a4f2-4531-b982-e1b783f83677 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:20.319378 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b5506-cca2-4c90-a419-1fc9ed7e1bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:20.320509 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b5506-cca2-4c90-a419-1fc9ed7e1bda None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:20.321017 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b5506-cca2-4c90-a419-1fc9ed7e1bda None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:20.321517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b5506-cca2-4c90-a419-1fc9ed7e1bda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:20.322549 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b5506-cca2-4c90-a419-1fc9ed7e1bda None admin] RBAC: Adding query filter params (role.id=c60a99de086f45c6820f7872a6b43feb, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=cbf513ee25374e49b3d8ce28843e393a) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:20.323065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b5506-cca2-4c90-a419-1fc9ed7e1bda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:20.324812 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b5506-cca2-4c90-a419-1fc9ed7e1bda None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:20.331896 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 445/873] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:56:20 2022] GET /identity/v3/role_assignments?user.id=cbf513ee25374e49b3d8ce28843e393a&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=c60a99de086f45c6820f7872a6b43feb => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:20.400461 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa85bdd-0499-428c-80e3-b8ebe7c55657 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:20.400787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa85bdd-0499-428c-80e3-b8ebe7c55657 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:20.401036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa85bdd-0499-428c-80e3-b8ebe7c55657 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:20.402770 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 429/874] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:20.408515 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f67fe-8b8f-48da-86aa-41ec7a65f443 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:20.408792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f67fe-8b8f-48da-86aa-41ec7a65f443 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:20.409087 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f67fe-8b8f-48da-86aa-41ec7a65f443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:20.427576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-df1f67fe-8b8f-48da-86aa-41ec7a65f443 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:20.428889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-df1f67fe-8b8f-48da-86aa-41ec7a65f443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:20.441669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:20.510699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 446/875] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:20.534307 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa85bdd-0499-428c-80e3-b8ebe7c55657 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:20.642313 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa85bdd-0499-428c-80e3-b8ebe7c55657 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:20.677151 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c13e6-d24e-4e61-b5aa-6445d940e632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:20.677772 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c13e6-d24e-4e61-b5aa-6445d940e632 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:20.678069 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c13e6-d24e-4e61-b5aa-6445d940e632 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:20.678339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c13e6-d24e-4e61-b5aa-6445d940e632 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:20.679166 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c13e6-d24e-4e61-b5aa-6445d940e632 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:20.680613 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c13e6-d24e-4e61-b5aa-6445d940e632 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:20.697551 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 430/876] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Jul 18 13:56:20 2022] POST /identity/v3/services => generated 269 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:56:21.883780 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e41d18-cf99-4f63-a9be-ccecf6851234 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:21.884325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e41d18-cf99-4f63-a9be-ccecf6851234 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:21.884434 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e41d18-cf99-4f63-a9be-ccecf6851234 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:21.885950 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 447/877] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:21.891347 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7b33a-f481-4906-a5a7-834557165b5f None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:21.891780 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7b33a-f481-4906-a5a7-834557165b5f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:21.891977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7b33a-f481-4906-a5a7-834557165b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:21.909143 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b9d7b33a-f481-4906-a5a7-834557165b5f None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:21.910327 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b9d7b33a-f481-4906-a5a7-834557165b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:21.933622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:21.950550 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c1feb-1291-4ab5-a15f-6f43f1975155 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:21.950846 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c1feb-1291-4ab5-a15f-6f43f1975155 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:21.951128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c1feb-1291-4ab5-a15f-6f43f1975155 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:21.955994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 448/878] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:21.957852 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf95bff-fe2b-46fa-a77c-fc3fcb2ac345 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:21.958247 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf95bff-fe2b-46fa-a77c-fc3fcb2ac345 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:21.958625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf95bff-fe2b-46fa-a77c-fc3fcb2ac345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:21.977349 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-aaf95bff-fe2b-46fa-a77c-fc3fcb2ac345 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:21.978966 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-aaf95bff-fe2b-46fa-a77c-fc3fcb2ac345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:21.998569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 431/879] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:22.023235 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7b33a-f481-4906-a5a7-834557165b5f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:22.072533 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 449/880] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:22.094431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf95bff-fe2b-46fa-a77c-fc3fcb2ac345 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:22.137142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d7b33a-f481-4906-a5a7-834557165b5f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:22.173393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8160f-8a37-491d-9759-d9af6ad6c8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:22.174580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8160f-8a37-491d-9759-d9af6ad6c8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:22.175074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8160f-8a37-491d-9759-d9af6ad6c8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:22.175555 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8160f-8a37-491d-9759-d9af6ad6c8b4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:22.184328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:22.187193 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8160f-8a37-491d-9759-d9af6ad6c8b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:56:22.192956 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf95bff-fe2b-46fa-a77c-fc3fcb2ac345 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:22.197246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-eba8160f-8a37-491d-9759-d9af6ad6c8b4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 13:56:22.197246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:22.197246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:22.197246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:22.197246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:22.197246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:22.197246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:22.197246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:22.197246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:22.197246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:22.197246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:22.197246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:22.197246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:56:22.197246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:56:22.197246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:56:22.197246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:56:22.198171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:56:22.198171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:56:22.198171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:56:22.198171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:56:22.198171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:22.198171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:22.198171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:22.198171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:22.198171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:22.198171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:22.198171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:22.198171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:22.198171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:22.198171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:22.198171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:22.198171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:22.199022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:22.199022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:22.199022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:22.199022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:22.199022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:56:22.199022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:56:22.199022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:56:22.199022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:56:22.199022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:56:22.199022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:56:22.199022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 13:56:22.199022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:22.207526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 432/881] 10.222.0.94 () {56 vars in 1120 bytes} [Mon Jul 18 13:56:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:22.208616 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8160f-8a37-491d-9759-d9af6ad6c8b4 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:22.227610 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:22.232610 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a04663-d653-4baf-a81b-73db5d6cc1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:22.233623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a04663-d653-4baf-a81b-73db5d6cc1ac None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:22.233755 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a04663-d653-4baf-a81b-73db5d6cc1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:22.233949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a04663-d653-4baf-a81b-73db5d6cc1ac None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:22.234600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a04663-d653-4baf-a81b-73db5d6cc1ac None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:22.235983 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a04663-d653-4baf-a81b-73db5d6cc1ac None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:22.247844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-62a04663-d653-4baf-a81b-73db5d6cc1ac None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 13:56:22.247844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:22.247844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:56:22.247844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:56:22.247844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:56:22.247844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:56:22.247844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:56:22.247844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:22.247844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 13:56:22.247844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:22.247844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:22.247844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:22.247844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:22.247844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:22.247844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:22.247844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:22.247844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:22.248873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:22.248873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:22.248873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:22.248873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:22.248873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:22.248873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:22.248873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:56:22.248873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:56:22.248873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:56:22.248873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:56:22.248873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:22.248873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:22.248873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:22.248873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:22.248873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:22.248873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:22.249710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:22.249710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:22.249710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:22.249710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:22.249710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:22.249710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:22.249710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:22.249710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:22.249710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:22.249710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:22.249710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:56:22.249710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:22.249710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 13:56:22.249710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:22.249710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 450/882] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Jul 18 13:56:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:22.256522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a04663-d653-4baf-a81b-73db5d6cc1ac None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:22.345701 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8160f-8a37-491d-9759-d9af6ad6c8b4 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:22.392437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079ff95-484f-4e0c-a98b-610c9dd43a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:22.393398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a04663-d653-4baf-a81b-73db5d6cc1ac None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:22.393739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079ff95-484f-4e0c-a98b-610c9dd43a79 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:22.394261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079ff95-484f-4e0c-a98b-610c9dd43a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:22.395024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079ff95-484f-4e0c-a98b-610c9dd43a79 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:22.397056 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079ff95-484f-4e0c-a98b-610c9dd43a79 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:22.397649 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079ff95-484f-4e0c-a98b-610c9dd43a79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:22.399249 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079ff95-484f-4e0c-a98b-610c9dd43a79 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:22.416718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 433/883] 10.222.0.94 () {56 vars in 1115 bytes} [Mon Jul 18 13:56:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:22.418210 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079ff95-484f-4e0c-a98b-610c9dd43a79 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:22.430820 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fecaba-9a60-4666-bf32-39a6fb723690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:22.431484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fecaba-9a60-4666-bf32-39a6fb723690 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:22.431767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fecaba-9a60-4666-bf32-39a6fb723690 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:22.432086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fecaba-9a60-4666-bf32-39a6fb723690 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:22.432889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fecaba-9a60-4666-bf32-39a6fb723690 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:22.433097 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fecaba-9a60-4666-bf32-39a6fb723690 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:22.434476 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fecaba-9a60-4666-bf32-39a6fb723690 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:22.440051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 451/884] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Jul 18 13:56:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:22.441511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f55f04-c2f7-4084-997e-ef683f5ef804 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:22.441801 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f55f04-c2f7-4084-997e-ef683f5ef804 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:22.442076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f55f04-c2f7-4084-997e-ef683f5ef804 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:22.443523 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 452/885] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:22.446901 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f55f04-c2f7-4084-997e-ef683f5ef804 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:22.450965 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:22.510879 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:22.535278 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079ff95-484f-4e0c-a98b-610c9dd43a79 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:22.561853 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f55f04-c2f7-4084-997e-ef683f5ef804 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:22.570407 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206f9df-acd0-40b1-9c08-6be0db909541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:22.571415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206f9df-acd0-40b1-9c08-6be0db909541 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:22.571853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206f9df-acd0-40b1-9c08-6be0db909541 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:22.572331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206f9df-acd0-40b1-9c08-6be0db909541 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:22.584556 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206f9df-acd0-40b1-9c08-6be0db909541 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:22.586525 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206f9df-acd0-40b1-9c08-6be0db909541 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:22.597998 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694fa0f8-f4e2-4d2c-95a9-34fe5a6a0d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:22.598666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fa0f8-f4e2-4d2c-95a9-34fe5a6a0d84 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:22.598954 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fa0f8-f4e2-4d2c-95a9-34fe5a6a0d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:22.599293 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694fa0f8-f4e2-4d2c-95a9-34fe5a6a0d84 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:22.600043 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694fa0f8-f4e2-4d2c-95a9-34fe5a6a0d84 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:22.600491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694fa0f8-f4e2-4d2c-95a9-34fe5a6a0d84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:22.601918 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694fa0f8-f4e2-4d2c-95a9-34fe5a6a0d84 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:22.603135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-8206f9df-acd0-40b1-9c08-6be0db909541 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:22.603135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:22.603135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:22.603135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:22.603135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:22.603135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:22.603135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:22.603135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:22.603135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:22.603135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:22.603135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:22.603135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:22.603135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:22.603135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:22.603135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:22.604237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:22.604237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:22.604237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:22.604237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:22.604237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:22.604237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:22.604237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:22.604237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:22.604237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:22.604237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:22.604237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:22.604237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:22.604237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:22.604237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:22.604237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:22.604237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:22.604237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:22.605149 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:22.605149 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:22.605149 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:22.605149 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:22.605149 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 434/886] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:22.605516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cae487-ffde-47ac-a277-0eaa57485d6c None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:22.605801 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cae487-ffde-47ac-a277-0eaa57485d6c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:22.606077 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cae487-ffde-47ac-a277-0eaa57485d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:22.613664 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 453/887] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Jul 18 13:56:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:22.614721 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694fa0f8-f4e2-4d2c-95a9-34fe5a6a0d84 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:22.623730 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-53cae487-ffde-47ac-a277-0eaa57485d6c None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:22.624931 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-53cae487-ffde-47ac-a277-0eaa57485d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:22.686585 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:22.699982 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 435/888] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:22.699982 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cae487-ffde-47ac-a277-0eaa57485d6c None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:22.728040 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694fa0f8-f4e2-4d2c-95a9-34fe5a6a0d84 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:22.763014 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60922b22-d0ac-4bb8-b0e2-83a02abd6d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:22.763832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60922b22-d0ac-4bb8-b0e2-83a02abd6d8f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:22.764240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60922b22-d0ac-4bb8-b0e2-83a02abd6d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:22.764530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60922b22-d0ac-4bb8-b0e2-83a02abd6d8f None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:22.765262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60922b22-d0ac-4bb8-b0e2-83a02abd6d8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:22.765574 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60922b22-d0ac-4bb8-b0e2-83a02abd6d8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:22.766177 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:22.767080 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60922b22-d0ac-4bb8-b0e2-83a02abd6d8f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:22.779549 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 454/889] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:22.780572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60922b22-d0ac-4bb8-b0e2-83a02abd6d8f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:22.809166 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cae487-ffde-47ac-a277-0eaa57485d6c None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:22.844071 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52048d7-721e-44c1-aee7-fcb398cd53d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:22.844729 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52048d7-721e-44c1-aee7-fcb398cd53d7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:22.845017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52048d7-721e-44c1-aee7-fcb398cd53d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:22.845294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52048d7-721e-44c1-aee7-fcb398cd53d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:22.846087 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52048d7-721e-44c1-aee7-fcb398cd53d7 None admin] RBAC: Adding query filter params (interface=public, service_id=84156fa3f9f1457eaeed83eba5ba06a7, region_id=RegionOne) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:22.846398 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52048d7-721e-44c1-aee7-fcb398cd53d7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:22.847759 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52048d7-721e-44c1-aee7-fcb398cd53d7 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:22.852775 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 436/890] 10.222.0.94 () {56 vars in 1251 bytes} [Mon Jul 18 13:56:22 2022] GET /identity/v3/endpoints?service_id=84156fa3f9f1457eaeed83eba5ba06a7&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:22.854006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce2201-e185-41f4-8869-553ba6e62337 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:22.854297 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce2201-e185-41f4-8869-553ba6e62337 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:22.854574 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce2201-e185-41f4-8869-553ba6e62337 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:22.855804 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 437/891] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:22.856526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce2201-e185-41f4-8869-553ba6e62337 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:22.883277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60922b22-d0ac-4bb8-b0e2-83a02abd6d8f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:22.909198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:22.917502 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7a67b-cd8c-465b-bf5c-e0e349e809e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:22.918308 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7a67b-cd8c-465b-bf5c-e0e349e809e1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:22.918601 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7a67b-cd8c-465b-bf5c-e0e349e809e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:22.918885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7a67b-cd8c-465b-bf5c-e0e349e809e1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:22.919540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7a67b-cd8c-465b-bf5c-e0e349e809e1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:22.920953 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7a67b-cd8c-465b-bf5c-e0e349e809e1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:22.932727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-0ae7a67b-cd8c-465b-bf5c-e0e349e809e1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 13:56:22.932727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:22.932727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:56:22.932727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:56:22.932727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:56:22.932727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:56:22.932727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:56:22.932727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:22.932727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 13:56:22.932727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:22.932727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:22.932727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:22.932727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:22.932727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:22.932727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:22.932727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:22.932727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:22.933855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:22.933855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:22.933855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:22.933855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:22.933855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:22.933855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:22.933855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:56:22.933855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:56:22.933855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:56:22.933855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:56:22.933855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:22.933855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:22.933855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:22.933855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:22.933855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:22.933855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:22.934712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:22.934712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:22.934712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:22.934712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:22.934712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:22.934712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:22.934712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:22.934712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:22.934712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:22.934712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:22.934712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:56:22.934712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:22.934712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 13:56:22.934712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:22.934712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 455/892] 10.222.0.94 () {56 vars in 1126 bytes} [Mon Jul 18 13:56:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:22.935535 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7a67b-cd8c-465b-bf5c-e0e349e809e1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:22.965049 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce2201-e185-41f4-8869-553ba6e62337 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:22.987704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:23.000404 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85b94c-f6cc-4a14-bca9-0de200a111bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:23.001220 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85b94c-f6cc-4a14-bca9-0de200a111bb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:23.001712 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85b94c-f6cc-4a14-bca9-0de200a111bb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:23.002005 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85b94c-f6cc-4a14-bca9-0de200a111bb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:23.010170 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85b94c-f6cc-4a14-bca9-0de200a111bb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:23.011631 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85b94c-f6cc-4a14-bca9-0de200a111bb None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:23.027006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-6f85b94c-f6cc-4a14-bca9-0de200a111bb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 13:56:23.027006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:23.027006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:23.027006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:23.027006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:23.027006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:23.027006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:23.027006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:23.027006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:23.027006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:23.027006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:23.027006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:23.027006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:56:23.027006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:56:23.027006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:56:23.027006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:56:23.028090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:23.028090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:23.028090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:56:23.028090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:23.028090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:56:23.028090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:23.028090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:23.028090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:23.028090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:23.028090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:23.028090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:23.028090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:23.028090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:23.028090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:23.028090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:23.028090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:23.028090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:23.028090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:23.029058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:23.029058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:23.029058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:56:23.029058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:56:23.029058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:56:23.029058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:56:23.029058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:56:23.029058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:56:23.029058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 13:56:23.029058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:23.029058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 438/893] 10.222.0.94 () {56 vars in 1117 bytes} [Mon Jul 18 13:56:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:23.029652 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fad9a5-e9fc-45d0-b315-8689075fcabc None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:23.029874 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fad9a5-e9fc-45d0-b315-8689075fcabc None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:23.030081 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fad9a5-e9fc-45d0-b315-8689075fcabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:23.043601 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7a67b-cd8c-465b-bf5c-e0e349e809e1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:23.045832 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-10fad9a5-e9fc-45d0-b315-8689075fcabc None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:23.047017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-10fad9a5-e9fc-45d0-b315-8689075fcabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:23.077076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9164479-4606-492e-942a-c1c224f5f795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:23.077735 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9164479-4606-492e-942a-c1c224f5f795 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:23.078028 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9164479-4606-492e-942a-c1c224f5f795 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:23.078318 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9164479-4606-492e-942a-c1c224f5f795 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:23.079273 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9164479-4606-492e-942a-c1c224f5f795 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:23.080675 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9164479-4606-492e-942a-c1c224f5f795 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:23.086815 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 456/894] 10.222.0.94 () {56 vars in 1087 bytes} [Mon Jul 18 13:56:22 2022] GET /identity/v3/services => generated 1268 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 13:56:23.087590 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9164479-4606-492e-942a-c1c224f5f795 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:23.129038 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 439/895] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:23.130371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fad9a5-e9fc-45d0-b315-8689075fcabc None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:23.161401 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:23.211601 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9164479-4606-492e-942a-c1c224f5f795 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:23.252026 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c217813-f941-4608-84e4-21f7aab38244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:23.252712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c217813-f941-4608-84e4-21f7aab38244 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:23.253058 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c217813-f941-4608-84e4-21f7aab38244 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:23.253344 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c217813-f941-4608-84e4-21f7aab38244 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:23.254064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c217813-f941-4608-84e4-21f7aab38244 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:23.254380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c217813-f941-4608-84e4-21f7aab38244 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:23.255822 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c217813-f941-4608-84e4-21f7aab38244 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:23.258746 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fad9a5-e9fc-45d0-b315-8689075fcabc None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:23.262333 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 457/896] 10.222.0.94 () {56 vars in 1122 bytes} [Mon Jul 18 13:56:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:23.263204 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c217813-f941-4608-84e4-21f7aab38244 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:23.277530 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:23.299703 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a04e9-50ee-4f06-98cd-a89e7b015747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:23.300417 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a04e9-50ee-4f06-98cd-a89e7b015747 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:23.300706 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a04e9-50ee-4f06-98cd-a89e7b015747 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:23.300991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a04e9-50ee-4f06-98cd-a89e7b015747 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:23.301836 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a04e9-50ee-4f06-98cd-a89e7b015747 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:23.302153 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a04e9-50ee-4f06-98cd-a89e7b015747 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:23.304134 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a04e9-50ee-4f06-98cd-a89e7b015747 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:23.323040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 440/897] 10.222.0.94 () {56 vars in 1149 bytes} [Mon Jul 18 13:56:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:23.323787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a04e9-50ee-4f06-98cd-a89e7b015747 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:23.377759 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c217813-f941-4608-84e4-21f7aab38244 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:23.398300 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:23.414238 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21716a5-32e5-400f-bdc5-f83c75f45df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:23.415049 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21716a5-32e5-400f-bdc5-f83c75f45df4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:23.415338 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21716a5-32e5-400f-bdc5-f83c75f45df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:23.415614 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21716a5-32e5-400f-bdc5-f83c75f45df4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:23.416303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21716a5-32e5-400f-bdc5-f83c75f45df4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:23.417682 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21716a5-32e5-400f-bdc5-f83c75f45df4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:23.430758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-d21716a5-32e5-400f-bdc5-f83c75f45df4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 13:56:23.430758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:23.430758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:56:23.430758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:56:23.430758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:56:23.430758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:56:23.430758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:56:23.430758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:23.430758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 13:56:23.430758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:23.430758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:23.430758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:23.430758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:23.430758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:23.430758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:23.430758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:23.431730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:23.431730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:23.431730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:23.431730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:23.431730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:23.431730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:23.431730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:23.431730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:56:23.431730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:56:23.431730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:56:23.431730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:56:23.431730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:23.431730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:23.431730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:23.431730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:23.431730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:23.437952 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:23.437952 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:23.437952 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:23.437952 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:23.437952 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:23.437952 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:23.437952 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:23.437952 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:23.437952 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:23.437952 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:23.437952 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:23.437952 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:56:23.437952 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:23.437952 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 13:56:23.437952 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:23.438957 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a04e9-50ee-4f06-98cd-a89e7b015747 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:23.439050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 458/898] 10.222.0.94 () {56 vars in 1132 bytes} [Mon Jul 18 13:56:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:23.439050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21716a5-32e5-400f-bdc5-f83c75f45df4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:23.473338 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b40700-aac8-4fbc-bbfc-ed81416210db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:23.474017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b40700-aac8-4fbc-bbfc-ed81416210db None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:23.474381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b40700-aac8-4fbc-bbfc-ed81416210db None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:23.474735 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b40700-aac8-4fbc-bbfc-ed81416210db None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:23.476173 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b40700-aac8-4fbc-bbfc-ed81416210db None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:23.476489 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b40700-aac8-4fbc-bbfc-ed81416210db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:23.477858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b40700-aac8-4fbc-bbfc-ed81416210db None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:23.491978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 441/899] 10.222.0.94 () {56 vars in 1122 bytes} [Mon Jul 18 13:56:23 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:23.491978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b40700-aac8-4fbc-bbfc-ed81416210db None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:23.530225 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:23.568601 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21716a5-32e5-400f-bdc5-f83c75f45df4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:23.616035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b40700-aac8-4fbc-bbfc-ed81416210db None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:23.621471 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1dd6bc-dc87-40ee-a039-af86ab760a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:23.622157 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1dd6bc-dc87-40ee-a039-af86ab760a98 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:23.622442 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1dd6bc-dc87-40ee-a039-af86ab760a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:23.622725 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1dd6bc-dc87-40ee-a039-af86ab760a98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:23.655843 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f8969-4b56-4bd4-a0a2-cf0b867d0ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:23.656551 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f8969-4b56-4bd4-a0a2-cf0b867d0ffc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:23.656856 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f8969-4b56-4bd4-a0a2-cf0b867d0ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:23.657110 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f8969-4b56-4bd4-a0a2-cf0b867d0ffc None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:23.657815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f8969-4b56-4bd4-a0a2-cf0b867d0ffc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:23.658126 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f8969-4b56-4bd4-a0a2-cf0b867d0ffc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:23.659488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f8969-4b56-4bd4-a0a2-cf0b867d0ffc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:23.664920 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 442/900] 10.222.0.94 () {56 vars in 1126 bytes} [Mon Jul 18 13:56:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:23.670543 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f8969-4b56-4bd4-a0a2-cf0b867d0ffc None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:23.702684 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:23.726625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1dd6bc-dc87-40ee-a039-af86ab760a98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:23.728270 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1dd6bc-dc87-40ee-a039-af86ab760a98 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:23.783396 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:23.787185 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f8969-4b56-4bd4-a0a2-cf0b867d0ffc None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:23.824424 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333c70f-cdf5-4e40-b5c7-69a85a20a81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:23.825207 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333c70f-cdf5-4e40-b5c7-69a85a20a81c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:23.825560 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333c70f-cdf5-4e40-b5c7-69a85a20a81c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:23.825931 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333c70f-cdf5-4e40-b5c7-69a85a20a81c None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:23.826711 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333c70f-cdf5-4e40-b5c7-69a85a20a81c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:23.827088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333c70f-cdf5-4e40-b5c7-69a85a20a81c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:23.828559 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333c70f-cdf5-4e40-b5c7-69a85a20a81c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:23.834231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 443/901] 10.222.0.94 () {56 vars in 1126 bytes} [Mon Jul 18 13:56:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:23.839876 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333c70f-cdf5-4e40-b5c7-69a85a20a81c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:23.844227 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 459/902] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:56:23 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 13:56:23.849426 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1dd6bc-dc87-40ee-a039-af86ab760a98 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:23.945530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:23.964629 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1dd6bc-dc87-40ee-a039-af86ab760a98 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:23.967903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333c70f-cdf5-4e40-b5c7-69a85a20a81c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:24.002641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14c0a9-b70d-4713-92cf-69b94ef232d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:24.003389 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14c0a9-b70d-4713-92cf-69b94ef232d7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:24.003684 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14c0a9-b70d-4713-92cf-69b94ef232d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:24.004076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14c0a9-b70d-4713-92cf-69b94ef232d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:24.010919 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce2466-dfb2-4d40-ae6e-ffb4ea285d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:24.012252 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce2466-dfb2-4d40-ae6e-ffb4ea285d7f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:24.012787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce2466-dfb2-4d40-ae6e-ffb4ea285d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:24.013651 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce2466-dfb2-4d40-ae6e-ffb4ea285d7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:24.014923 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce2466-dfb2-4d40-ae6e-ffb4ea285d7f None admin] RBAC: Adding query filter params (interface=public, service_id=17603a2a92c547809e8e8c2eab72b506, region_id=RegionOne) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:24.015101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df14c0a9-b70d-4713-92cf-69b94ef232d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:24.015600 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce2466-dfb2-4d40-ae6e-ffb4ea285d7f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:24.016605 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df14c0a9-b70d-4713-92cf-69b94ef232d7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:24.017630 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce2466-dfb2-4d40-ae6e-ffb4ea285d7f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:24.024301 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 444/903] 10.222.0.94 () {56 vars in 1251 bytes} [Mon Jul 18 13:56:23 2022] GET /identity/v3/endpoints?service_id=17603a2a92c547809e8e8c2eab72b506&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:24.028844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce2466-dfb2-4d40-ae6e-ffb4ea285d7f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:24.035101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-df14c0a9-b70d-4713-92cf-69b94ef232d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:24.035101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:24.035101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:24.035101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:24.035101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:24.035101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:24.035101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:24.035101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:24.035101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:24.035101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:24.035101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:24.035101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:24.035101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:24.035101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:24.035101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:24.036020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:24.036020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:24.036020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:24.036020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:24.036020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:24.036020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:24.036020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:24.036020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:24.036020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:24.036020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:24.036020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:24.036020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:24.036020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:24.036020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:24.036020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:24.036020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:24.036020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:24.036939 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:24.036939 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:24.036939 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:24.036939 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:24.040286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 460/904] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:24.042454 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14c0a9-b70d-4713-92cf-69b94ef232d7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:24.063352 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:24.155598 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce2466-dfb2-4d40-ae6e-ffb4ea285d7f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:24.158150 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14c0a9-b70d-4713-92cf-69b94ef232d7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:24.196677 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3568bd-c72c-4090-a6b9-4afd2e3fdbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:24.197996 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3568bd-c72c-4090-a6b9-4afd2e3fdbe8 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:24.198737 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3568bd-c72c-4090-a6b9-4afd2e3fdbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:24.199618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3568bd-c72c-4090-a6b9-4afd2e3fdbe8 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:24.200803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3568bd-c72c-4090-a6b9-4afd2e3fdbe8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:24.203076 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3568bd-c72c-4090-a6b9-4afd2e3fdbe8 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:24.207735 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93334c6-52f2-48ec-a638-c004202c6980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:24.208465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93334c6-52f2-48ec-a638-c004202c6980 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:24.208754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93334c6-52f2-48ec-a638-c004202c6980 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:24.209038 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93334c6-52f2-48ec-a638-c004202c6980 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:24.210431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93334c6-52f2-48ec-a638-c004202c6980 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:24.210751 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93334c6-52f2-48ec-a638-c004202c6980 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:24.212345 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93334c6-52f2-48ec-a638-c004202c6980 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:24.212604 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 445/905] 10.222.0.94 () {56 vars in 1087 bytes} [Mon Jul 18 13:56:24 2022] GET /identity/v3/services => generated 1268 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 13:56:24.215904 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:24.232689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 461/906] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:24.238648 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3568bd-c72c-4090-a6b9-4afd2e3fdbe8 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:24.344712 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:24.355413 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3568bd-c72c-4090-a6b9-4afd2e3fdbe8 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:24.395260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b889f9-09ef-4adb-add6-f4fe1065fca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:24.396094 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b889f9-09ef-4adb-add6-f4fe1065fca3 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:24.396434 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b889f9-09ef-4adb-add6-f4fe1065fca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:24.396717 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b889f9-09ef-4adb-add6-f4fe1065fca3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:24.403593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b889f9-09ef-4adb-add6-f4fe1065fca3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:24.405071 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b889f9-09ef-4adb-add6-f4fe1065fca3 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:24.419323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-e2b889f9-09ef-4adb-add6-f4fe1065fca3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:24.419323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:24.419323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:24.419323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:24.419323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:24.419323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:24.419323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:24.419323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:24.419323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:24.419323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:24.419323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:24.419323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:24.419323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:56:24.419323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:56:24.419323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:56:24.420242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:56:24.420242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:24.420242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:24.420242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:24.420242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:24.420242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:24.420242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:24.420242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:24.420242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:24.420242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:24.420242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:24.420242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:24.420242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:24.420242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:24.420242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:24.420242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:24.420242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:24.421150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:56:24.421150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:56:24.421150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:56:24.421150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:56:24.421150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:56:24.421150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:56:24.421150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:24.421150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:24.421150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 446/907] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:56:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:24.426250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93334c6-52f2-48ec-a638-c004202c6980 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:24.538323 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93334c6-52f2-48ec-a638-c004202c6980 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:24.574857 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe6a1d-e9aa-4a08-82df-5838c1da8bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:24.575581 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe6a1d-e9aa-4a08-82df-5838c1da8bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:24.575879 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe6a1d-e9aa-4a08-82df-5838c1da8bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:24.576193 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe6a1d-e9aa-4a08-82df-5838c1da8bd2 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:24.576998 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe6a1d-e9aa-4a08-82df-5838c1da8bd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:24.577579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe6a1d-e9aa-4a08-82df-5838c1da8bd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:24.578777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe6a1d-e9aa-4a08-82df-5838c1da8bd2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:24.592709 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 462/908] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:56:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:24.598492 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b889f9-09ef-4adb-add6-f4fe1065fca3 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:24.708499 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b889f9-09ef-4adb-add6-f4fe1065fca3 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:24.745396 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3771859-02a5-4fc8-89d2-bd6bd22e5952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:24.746449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3771859-02a5-4fc8-89d2-bd6bd22e5952 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:24.746951 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3771859-02a5-4fc8-89d2-bd6bd22e5952 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:24.747427 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3771859-02a5-4fc8-89d2-bd6bd22e5952 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:24.748492 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3771859-02a5-4fc8-89d2-bd6bd22e5952 None admin] RBAC: Adding query filter params (role.id=039f54fd5a574af19dad80e091f25108, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=94d824bffc2c4af299c537056bc21734) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:24.749023 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3771859-02a5-4fc8-89d2-bd6bd22e5952 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:24.750669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3771859-02a5-4fc8-89d2-bd6bd22e5952 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:24.757186 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 447/909] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:56:24 2022] GET /identity/v3/role_assignments?user.id=94d824bffc2c4af299c537056bc21734&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=039f54fd5a574af19dad80e091f25108 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:25.142870 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36a90c-2234-421a-a583-088801d9e72a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:25.143408 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36a90c-2234-421a-a583-088801d9e72a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:25.143596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36a90c-2234-421a-a583-088801d9e72a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:25.145095 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 463/910] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:25.150336 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7ae7b-7ff3-483b-8880-57b63a0a5288 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:25.151130 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7ae7b-7ff3-483b-8880-57b63a0a5288 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:25.151642 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7ae7b-7ff3-483b-8880-57b63a0a5288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:25.169475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-bda7ae7b-7ff3-483b-8880-57b63a0a5288 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:25.171002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bda7ae7b-7ff3-483b-8880-57b63a0a5288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:25.181928 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:25.243728 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 448/911] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:25.265893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36a90c-2234-421a-a583-088801d9e72a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:25.327407 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:25.379356 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36a90c-2234-421a-a583-088801d9e72a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:25.416518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642cb97-e984-4511-8f00-42bf43ad5f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:25.417376 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642cb97-e984-4511-8f00-42bf43ad5f12 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:25.417666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642cb97-e984-4511-8f00-42bf43ad5f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:25.417948 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642cb97-e984-4511-8f00-42bf43ad5f12 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:25.418616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642cb97-e984-4511-8f00-42bf43ad5f12 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:25.420083 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642cb97-e984-4511-8f00-42bf43ad5f12 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:25.432635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-1642cb97-e984-4511-8f00-42bf43ad5f12 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 13:56:25.432635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:25.432635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:56:25.432635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:56:25.432635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:56:25.432635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:56:25.432635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:56:25.432635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:25.432635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 13:56:25.432635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:25.432635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:25.432635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:25.432635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:25.432635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:25.432635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:25.432635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:25.432635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:25.433702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:25.433702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:25.433702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:25.433702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:25.433702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:25.433702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:25.433702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:56:25.433702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:56:25.433702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:56:25.433702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:56:25.433702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:25.433702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:25.433702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:25.433702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:25.433702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:25.433702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:25.434552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:25.434552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:25.434552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:25.434552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:25.434552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:25.434552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:25.434552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:25.434552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:25.434552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:25.434552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:25.434552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:56:25.434552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:25.434552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 13:56:25.434552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:25.434552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 464/912] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Jul 18 13:56:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:25.440522 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7ae7b-7ff3-483b-8880-57b63a0a5288 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:25.543590 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7ae7b-7ff3-483b-8880-57b63a0a5288 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:25.579647 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7c8d8-16b0-472b-a031-9c9e78ce4208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:25.580681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7c8d8-16b0-472b-a031-9c9e78ce4208 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:25.581221 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7c8d8-16b0-472b-a031-9c9e78ce4208 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:25.581698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7c8d8-16b0-472b-a031-9c9e78ce4208 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:25.582599 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7c8d8-16b0-472b-a031-9c9e78ce4208 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:25.583115 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7c8d8-16b0-472b-a031-9c9e78ce4208 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:25.584733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7c8d8-16b0-472b-a031-9c9e78ce4208 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:25.590085 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 449/913] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Jul 18 13:56:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:25.595381 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642cb97-e984-4511-8f00-42bf43ad5f12 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:25.640106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42be53-2f4e-4624-9b54-791dd36ce78b None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:25.640682 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42be53-2f4e-4624-9b54-791dd36ce78b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:25.641171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42be53-2f4e-4624-9b54-791dd36ce78b None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:25.642881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 450/914] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:56:25.648681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64301da1-e7e9-4e4f-9b9c-5a62f3a7700f None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:25.649200 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64301da1-e7e9-4e4f-9b9c-5a62f3a7700f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:25.649698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64301da1-e7e9-4e4f-9b9c-5a62f3a7700f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:25.666682 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-64301da1-e7e9-4e4f-9b9c-5a62f3a7700f None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:25.667979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-64301da1-e7e9-4e4f-9b9c-5a62f3a7700f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:25.670791 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:25.706374 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642cb97-e984-4511-8f00-42bf43ad5f12 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:25.731457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:25.750379 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 451/915] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 13:56:25.754155 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a69c0b-2f9b-4848-93fe-ea138ad91f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:25.754831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a69c0b-2f9b-4848-93fe-ea138ad91f24 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:25.755217 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a69c0b-2f9b-4848-93fe-ea138ad91f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:25.755508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a69c0b-2f9b-4848-93fe-ea138ad91f24 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:25.756264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a69c0b-2f9b-4848-93fe-ea138ad91f24 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:25.756651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a69c0b-2f9b-4848-93fe-ea138ad91f24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:25.758105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a69c0b-2f9b-4848-93fe-ea138ad91f24 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:25.763717 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 465/916] 10.222.0.94 () {56 vars in 1114 bytes} [Mon Jul 18 13:56:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:25.767367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64301da1-e7e9-4e4f-9b9c-5a62f3a7700f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:25.773409 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a69c0b-2f9b-4848-93fe-ea138ad91f24 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:25.880674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64301da1-e7e9-4e4f-9b9c-5a62f3a7700f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:25.892844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a69c0b-2f9b-4848-93fe-ea138ad91f24 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:25.920214 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc15b3c-51e7-4b5f-91cd-818e8f431b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:25.921237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc15b3c-51e7-4b5f-91cd-818e8f431b23 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:25.921678 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc15b3c-51e7-4b5f-91cd-818e8f431b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:25.922160 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc15b3c-51e7-4b5f-91cd-818e8f431b23 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:25.923065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc15b3c-51e7-4b5f-91cd-818e8f431b23 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:25.924716 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc15b3c-51e7-4b5f-91cd-818e8f431b23 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:25.930278 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f101648-572f-4ce3-9c37-444807b503bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:25.931023 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f101648-572f-4ce3-9c37-444807b503bd None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:25.931468 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f101648-572f-4ce3-9c37-444807b503bd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:25.931758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f101648-572f-4ce3-9c37-444807b503bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:25.932634 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f101648-572f-4ce3-9c37-444807b503bd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:25.934058 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f101648-572f-4ce3-9c37-444807b503bd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:25.935502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:25.957214 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 452/917] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Jul 18 13:56:25 2022] POST /identity/v3/services => generated 241 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:56:25.961409 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 466/918] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Jul 18 13:56:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:56:26.369540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a037f-752e-413e-8807-563d371f0c2d None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:26.370027 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a037f-752e-413e-8807-563d371f0c2d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:26.370027 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a037f-752e-413e-8807-563d371f0c2d None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:26.371603 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 453/919] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:26.377099 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b872b-3f10-4b12-9a86-a7f06e719d02 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:26.377500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b872b-3f10-4b12-9a86-a7f06e719d02 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:26.377673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b872b-3f10-4b12-9a86-a7f06e719d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:26.394688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-427b872b-3f10-4b12-9a86-a7f06e719d02 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:26.395933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-427b872b-3f10-4b12-9a86-a7f06e719d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:26.432838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:26.474299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 467/920] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:26.496662 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a037f-752e-413e-8807-563d371f0c2d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:26.609122 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a037f-752e-413e-8807-563d371f0c2d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:26.648912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dcd766-53ef-42dc-8bc4-de4dd846bb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:26.650057 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcd766-53ef-42dc-8bc4-de4dd846bb4f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:26.650496 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcd766-53ef-42dc-8bc4-de4dd846bb4f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:26.650922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcd766-53ef-42dc-8bc4-de4dd846bb4f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:26.651747 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dcd766-53ef-42dc-8bc4-de4dd846bb4f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:26.653335 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dcd766-53ef-42dc-8bc4-de4dd846bb4f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:26.667597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-01dcd766-53ef-42dc-8bc4-de4dd846bb4f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 13:56:26.667597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:26.667597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:56:26.667597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:56:26.667597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:56:26.667597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:56:26.667597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:56:26.667597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:26.667597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 13:56:26.667597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:26.667597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:26.667597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:26.667597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:26.667597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:26.667597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:26.667597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:26.668850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:26.668850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:26.668850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:26.668850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:26.668850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:26.668850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:26.668850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:26.668850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:56:26.668850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:56:26.668850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:56:26.668850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:56:26.668850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:26.668850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:26.668850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:26.668850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:26.668850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:26.669912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:26.669912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:26.669912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:26.669912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:26.669912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:26.669912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:26.669912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:26.669912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:26.669912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:26.669912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:26.669912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:26.669912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:56:26.669912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:26.669912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 13:56:26.669912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:26.672513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 454/921] 10.222.0.94 () {56 vars in 1132 bytes} [Mon Jul 18 13:56:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:26.679995 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b872b-3f10-4b12-9a86-a7f06e719d02 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:26.783981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b872b-3f10-4b12-9a86-a7f06e719d02 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:26.820013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b0f20-10b4-4329-bad0-9afc8d651a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:26.820689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b0f20-10b4-4329-bad0-9afc8d651a38 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:26.820980 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b0f20-10b4-4329-bad0-9afc8d651a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:26.821262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b0f20-10b4-4329-bad0-9afc8d651a38 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:26.821970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b0f20-10b4-4329-bad0-9afc8d651a38 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:26.822282 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b0f20-10b4-4329-bad0-9afc8d651a38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:26.823660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b0f20-10b4-4329-bad0-9afc8d651a38 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:26.828828 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 468/922] 10.222.0.94 () {56 vars in 1126 bytes} [Mon Jul 18 13:56:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:26.835900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dcd766-53ef-42dc-8bc4-de4dd846bb4f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:26.899453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:26.929663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858020b9-eeec-413f-bb79-0d404416db09 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:26.929953 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858020b9-eeec-413f-bb79-0d404416db09 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:26.930237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858020b9-eeec-413f-bb79-0d404416db09 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:26.931602 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 469/923] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:56:26.940733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dcd766-53ef-42dc-8bc4-de4dd846bb4f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:26.941663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52155d16-bc60-4c23-a3a6-e54928a6c2e7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:26.941987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52155d16-bc60-4c23-a3a6-e54928a6c2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:26.942288 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52155d16-bc60-4c23-a3a6-e54928a6c2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:26.965917 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-52155d16-bc60-4c23-a3a6-e54928a6c2e7 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:26.967128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-52155d16-bc60-4c23-a3a6-e54928a6c2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:26.982562 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03455d6b-a3b4-41fc-8d80-8c1ce3dfff7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:26.983517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03455d6b-a3b4-41fc-8d80-8c1ce3dfff7c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:26.984121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03455d6b-a3b4-41fc-8d80-8c1ce3dfff7c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:26.984783 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03455d6b-a3b4-41fc-8d80-8c1ce3dfff7c None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:26.985684 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03455d6b-a3b4-41fc-8d80-8c1ce3dfff7c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:26.986186 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03455d6b-a3b4-41fc-8d80-8c1ce3dfff7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:26.987878 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03455d6b-a3b4-41fc-8d80-8c1ce3dfff7c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:26.991570 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:26.999540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 455/924] 10.222.0.94 () {56 vars in 1126 bytes} [Mon Jul 18 13:56:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:27.001980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03455d6b-a3b4-41fc-8d80-8c1ce3dfff7c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:27.049230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 470/925] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 13:56:27.072966 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52155d16-bc60-4c23-a3a6-e54928a6c2e7 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:27.113739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03455d6b-a3b4-41fc-8d80-8c1ce3dfff7c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:27.145807 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:27.155358 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10271428-3c08-4bcb-9a10-41672c372750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:27.156926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10271428-3c08-4bcb-9a10-41672c372750 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:27.157272 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10271428-3c08-4bcb-9a10-41672c372750 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:27.157563 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10271428-3c08-4bcb-9a10-41672c372750 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:27.158330 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10271428-3c08-4bcb-9a10-41672c372750 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:27.159815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10271428-3c08-4bcb-9a10-41672c372750 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:27.191968 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 456/926] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Jul 18 13:56:27 2022] POST /identity/v3/endpoints => generated 347 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:56:27.217033 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52155d16-bc60-4c23-a3a6-e54928a6c2e7 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:27.262930 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3212f3-b1ad-4c51-a831-0841a3bc5840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:27.263758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3212f3-b1ad-4c51-a831-0841a3bc5840 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:27.264078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3212f3-b1ad-4c51-a831-0841a3bc5840 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:27.264359 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3212f3-b1ad-4c51-a831-0841a3bc5840 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:27.276210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3212f3-b1ad-4c51-a831-0841a3bc5840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:56:27.284705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-fc3212f3-b1ad-4c51-a831-0841a3bc5840 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 13:56:27.284705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:27.284705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:27.284705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:27.284705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:27.284705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:27.284705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:27.284705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:27.284705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:27.284705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:27.284705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:27.284705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:27.284705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:56:27.284705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:56:27.284705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:56:27.284705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:56:27.285694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:56:27.285694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:56:27.285694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:56:27.285694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:56:27.285694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:27.285694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:27.285694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:27.285694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:27.285694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:27.285694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:27.285694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:27.285694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:27.285694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:27.285694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:27.285694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:27.285694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:27.286955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:27.286955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:27.286955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:27.286955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:27.286955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:56:27.286955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:56:27.286955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:56:27.286955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:56:27.286955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:56:27.286955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:56:27.286955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 13:56:27.286955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:27.286955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 471/927] 10.222.0.94 () {56 vars in 1120 bytes} [Mon Jul 18 13:56:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:27.293401 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10271428-3c08-4bcb-9a10-41672c372750 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:27.401669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10271428-3c08-4bcb-9a10-41672c372750 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:27.437095 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a92be0-09fc-4bf4-a55f-6823eedd53f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:27.437744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a92be0-09fc-4bf4-a55f-6823eedd53f0 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:27.438040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a92be0-09fc-4bf4-a55f-6823eedd53f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:27.438324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a92be0-09fc-4bf4-a55f-6823eedd53f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:27.439078 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a92be0-09fc-4bf4-a55f-6823eedd53f0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:27.439392 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a92be0-09fc-4bf4-a55f-6823eedd53f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:27.440798 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a92be0-09fc-4bf4-a55f-6823eedd53f0 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:27.449490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 457/928] 10.222.0.94 () {56 vars in 1115 bytes} [Mon Jul 18 13:56:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:27.455273 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3212f3-b1ad-4c51-a831-0841a3bc5840 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:27.474426 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:27.562188 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3212f3-b1ad-4c51-a831-0841a3bc5840 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:27.597117 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff9926-1b8f-45a0-b8f9-4ea66a64db76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:27.598283 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff9926-1b8f-45a0-b8f9-4ea66a64db76 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:27.598795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff9926-1b8f-45a0-b8f9-4ea66a64db76 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:27.599265 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff9926-1b8f-45a0-b8f9-4ea66a64db76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:27.611070 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff9926-1b8f-45a0-b8f9-4ea66a64db76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:27.612843 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff9926-1b8f-45a0-b8f9-4ea66a64db76 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:27.630315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-29ff9926-1b8f-45a0-b8f9-4ea66a64db76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:27.630315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:27.630315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:27.630315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:27.630315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:27.630315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:27.630315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:27.630315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:27.630315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:27.630315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:27.630315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:27.630315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:27.630315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:27.630315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:27.630315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:27.631252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:27.631252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:27.631252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:27.631252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:27.631252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:27.631252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:27.631252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:27.631252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:27.631252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:27.631252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:27.631252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:27.631252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:27.631252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:27.631252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:27.631252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:27.631252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:27.631252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:27.632816 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:27.632816 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:27.632816 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:27.632816 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:27.634697 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 472/929] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:27.640460 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a92be0-09fc-4bf4-a55f-6823eedd53f0 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:27.689904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:27.749761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a92be0-09fc-4bf4-a55f-6823eedd53f0 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:27.785240 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f336d-892a-4236-9045-e48ce5be297d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:27.785979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f336d-892a-4236-9045-e48ce5be297d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:27.786273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f336d-892a-4236-9045-e48ce5be297d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:27.786555 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f336d-892a-4236-9045-e48ce5be297d None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:27.787344 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f336d-892a-4236-9045-e48ce5be297d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:27.787701 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f336d-892a-4236-9045-e48ce5be297d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:27.789131 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f336d-892a-4236-9045-e48ce5be297d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:27.802289 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 458/930] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:27.808416 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff9926-1b8f-45a0-b8f9-4ea66a64db76 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:27.877974 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:27.914558 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff9926-1b8f-45a0-b8f9-4ea66a64db76 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:27.960511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ad439-bc4e-4cf2-9045-9c3bc4426bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:27.961666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ad439-bc4e-4cf2-9045-9c3bc4426bdb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:27.962150 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ad439-bc4e-4cf2-9045-9c3bc4426bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:27.962637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ad439-bc4e-4cf2-9045-9c3bc4426bdb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:27.970749 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ad439-bc4e-4cf2-9045-9c3bc4426bdb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:27.972468 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ad439-bc4e-4cf2-9045-9c3bc4426bdb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:27.987444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-0a3ad439-bc4e-4cf2-9045-9c3bc4426bdb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 13:56:27.987444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:27.987444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:27.987444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:27.987444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:27.987444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:27.987444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:27.987444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:27.987444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:27.987444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:27.987444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:27.987444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:27.987444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:56:27.987444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:56:27.987444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:56:27.987444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:56:27.988656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:27.988656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:27.988656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:56:27.988656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:27.988656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:56:27.988656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:27.988656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:27.988656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:27.988656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:27.988656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:27.988656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:27.988656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:27.988656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:27.988656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:27.988656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:27.988656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:27.988656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:27.988656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:27.989595 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:27.989595 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:27.989595 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:56:27.989595 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:56:27.989595 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:56:27.989595 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:56:27.989595 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:56:27.989595 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:56:27.989595 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 13:56:27.989595 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:27.992703 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 473/931] 10.222.0.94 () {56 vars in 1117 bytes} [Mon Jul 18 13:56:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:28.003813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f336d-892a-4236-9045-e48ce5be297d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:28.081731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd22d26-66f8-419a-8f56-5ad00996814c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:28.082019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd22d26-66f8-419a-8f56-5ad00996814c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:28.082299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd22d26-66f8-419a-8f56-5ad00996814c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:28.084253 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 474/932] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:56:28.088979 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412dc960-1be1-446e-98c6-6d71d777dc1a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:28.089321 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412dc960-1be1-446e-98c6-6d71d777dc1a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:28.089546 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412dc960-1be1-446e-98c6-6d71d777dc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:28.107037 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-412dc960-1be1-446e-98c6-6d71d777dc1a None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:28.108385 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-412dc960-1be1-446e-98c6-6d71d777dc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:28.112182 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f336d-892a-4236-9045-e48ce5be297d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:28.115222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:28.148615 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa06a47-cf18-4aa3-9d37-3cbf29d2b20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:28.149293 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa06a47-cf18-4aa3-9d37-3cbf29d2b20e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:28.149632 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa06a47-cf18-4aa3-9d37-3cbf29d2b20e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:28.149910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa06a47-cf18-4aa3-9d37-3cbf29d2b20e None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:28.150767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa06a47-cf18-4aa3-9d37-3cbf29d2b20e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:28.151079 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa06a47-cf18-4aa3-9d37-3cbf29d2b20e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:28.152583 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa06a47-cf18-4aa3-9d37-3cbf29d2b20e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:28.175493 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 459/933] 10.222.0.94 () {56 vars in 1149 bytes} [Mon Jul 18 13:56:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:28.175493 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81ad3f-2959-418d-a6ab-c229269b26ad None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:28.175493 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81ad3f-2959-418d-a6ab-c229269b26ad None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:28.175493 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81ad3f-2959-418d-a6ab-c229269b26ad None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:28.176203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 460/934] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:28.176888 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81ad3f-2959-418d-a6ab-c229269b26ad None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:28.191590 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 475/935] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 13:56:28.192860 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dc36e-b840-4750-bcbe-6a53a2ccfe7f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:28.193161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dc36e-b840-4750-bcbe-6a53a2ccfe7f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:28.193430 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7dc36e-b840-4750-bcbe-6a53a2ccfe7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:28.209735 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fc7dc36e-b840-4750-bcbe-6a53a2ccfe7f None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:28.210972 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fc7dc36e-b840-4750-bcbe-6a53a2ccfe7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:28.266533 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:28.281147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 476/936] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:28.291909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81ad3f-2959-418d-a6ab-c229269b26ad None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:28.292798 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7dc36e-b840-4750-bcbe-6a53a2ccfe7f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:28.328535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b0266-2edf-40fa-b1f6-ef6b30b39d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:28.329581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b0266-2edf-40fa-b1f6-ef6b30b39d00 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:28.329865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b0266-2edf-40fa-b1f6-ef6b30b39d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:28.330138 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b0266-2edf-40fa-b1f6-ef6b30b39d00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:28.340852 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b0266-2edf-40fa-b1f6-ef6b30b39d00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:28.342304 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b0266-2edf-40fa-b1f6-ef6b30b39d00 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:28.359547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-963b0266-2edf-40fa-b1f6-ef6b30b39d00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:28.359547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:28.359547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:28.359547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:28.359547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:28.359547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:28.359547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:28.359547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:28.359547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:28.359547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:28.359547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:28.359547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:28.359547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:28.359547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:28.359547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:28.360526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:28.360526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:28.360526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:28.360526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:28.360526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:28.360526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:28.360526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:28.360526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:28.360526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:28.360526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:28.360526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:28.360526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:28.360526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:28.360526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:28.360526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:28.360526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:28.360526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:28.366562 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:28.366562 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:28.366562 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:28.366562 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:28.366562 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 461/937] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:28.366562 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b0266-2edf-40fa-b1f6-ef6b30b39d00 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:28.382436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:28.415387 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7dc36e-b840-4750-bcbe-6a53a2ccfe7f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:28.452266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff3c2c-d60a-4952-b01d-7e98686d82e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:28.453077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff3c2c-d60a-4952-b01d-7e98686d82e9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:28.453368 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff3c2c-d60a-4952-b01d-7e98686d82e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:28.453649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff3c2c-d60a-4952-b01d-7e98686d82e9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:28.454558 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff3c2c-d60a-4952-b01d-7e98686d82e9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:28.455932 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff3c2c-d60a-4952-b01d-7e98686d82e9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:28.468770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-73ff3c2c-d60a-4952-b01d-7e98686d82e9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 13:56:28.468770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:28.468770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:56:28.468770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:56:28.468770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:56:28.468770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:56:28.468770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:56:28.468770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:28.468770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 13:56:28.468770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:28.468770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:28.468770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:28.468770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:28.468770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:28.468770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:28.468770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:28.468770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:28.469978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:28.469978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:28.469978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:28.469978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:28.469978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:28.469978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:28.469978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:56:28.469978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:56:28.469978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:56:28.469978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:56:28.469978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:28.469978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:28.469978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:28.469978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:28.469978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:28.469978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:28.470962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:28.470962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:28.470962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:28.470962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:28.470962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:28.470962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:28.470962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:28.470962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:28.470962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:28.470962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:28.470962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:56:28.470962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:28.470962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 13:56:28.470962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:28.470962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 477/938] 10.222.0.94 () {56 vars in 1126 bytes} [Mon Jul 18 13:56:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:28.471761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff3c2c-d60a-4952-b01d-7e98686d82e9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:28.483617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b0266-2edf-40fa-b1f6-ef6b30b39d00 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:28.520510 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bcde71-85cb-4ac3-9588-6b96a8bb9a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:28.521176 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bcde71-85cb-4ac3-9588-6b96a8bb9a58 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:28.521467 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bcde71-85cb-4ac3-9588-6b96a8bb9a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:28.521750 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bcde71-85cb-4ac3-9588-6b96a8bb9a58 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:28.522488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bcde71-85cb-4ac3-9588-6b96a8bb9a58 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:28.523934 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bcde71-85cb-4ac3-9588-6b96a8bb9a58 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:28.525315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: WARNING keystone.server.flask.common [None req-e1bcde71-85cb-4ac3-9588-6b96a8bb9a58 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 18 13:56:28.544010 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.sql.core [None req-e1bcde71-85cb-4ac3-9588-6b96a8bb9a58 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 13:56:28.544010 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 13:56:28.544010 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [parameters: {'id': '17afb59c607244c18cee8f98e39608bb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 13:56:28.544010 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80158) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 13:56:28.556621 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:28.561250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-e1bcde71-85cb-4ac3-9588-6b96a8bb9a58 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 17afb59c607244c18cee8f98e39608bb.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 17afb59c607244c18cee8f98e39608bb. Jul 18 13:56:28.561250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:28.561250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 13:56:28.561250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 13:56:28.561250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 13:56:28.561250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 13:56:28.561250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 13:56:28.561250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 13:56:28.561250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 13:56:28.561250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application conn.query(q) Jul 18 13:56:28.561250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 13:56:28.561250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 13:56:28.561250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 13:56:28.562106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application result.read() Jul 18 13:56:28.562106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 13:56:28.562106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 13:56:28.562106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 13:56:28.562106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 13:56:28.562106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 13:56:28.562106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 13:56:28.562106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 13:56:28.562106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 13:56:28.562106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 13:56:28.562106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:28.562106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 13:56:28.562106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:28.562106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:28.562106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 13:56:28.562106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 13:56:28.562106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 18 13:56:28.562106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 18 13:56:28.563083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 13:56:28.563083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application next(self.gen) Jul 18 13:56:28.563083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 13:56:28.563083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application yield resource Jul 18 13:56:28.563083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 13:56:28.563083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application next(self.gen) Jul 18 13:56:28.563083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 13:56:28.563083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 13:56:28.563083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 13:56:28.563083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 13:56:28.563083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 13:56:28.563083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise self.value Jul 18 13:56:28.563083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 13:56:28.563083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 13:56:28.563083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 13:56:28.563083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application session.commit() Jul 18 13:56:28.563083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 13:56:28.564324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 13:56:28.564324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 13:56:28.564324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 13:56:28.564324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 13:56:28.564324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self.session.flush() Jul 18 13:56:28.564324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 13:56:28.564324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 13:56:28.564324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 13:56:28.564324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 13:56:28.564324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 13:56:28.564324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application compat.raise_( Jul 18 13:56:28.564324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 13:56:28.564324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception Jul 18 13:56:28.564324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 13:56:28.564324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 13:56:28.564324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 13:56:28.564324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 13:56:28.565261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 13:56:28.565261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 13:56:28.565261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 13:56:28.565261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 13:56:28.565261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 13:56:28.565261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 13:56:28.565261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 13:56:28.565261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 13:56:28.565261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 13:56:28.565261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 13:56:28.565261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 13:56:28.565261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 13:56:28.565261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 13:56:28.565261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 13:56:28.565261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 13:56:28.566075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 13:56:28.566075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 13:56:28.566075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception Jul 18 13:56:28.566075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 13:56:28.566075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 13:56:28.566075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 13:56:28.566075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 13:56:28.566075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 13:56:28.566075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 13:56:28.566075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 13:56:28.566075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application conn.query(q) Jul 18 13:56:28.566075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 13:56:28.566075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 13:56:28.566075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 13:56:28.566075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application result.read() Jul 18 13:56:28.566075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 13:56:28.566075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 13:56:28.566973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 13:56:28.566973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 13:56:28.566973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 13:56:28.566973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 13:56:28.566973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 13:56:28.566973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 13:56:28.566973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 13:56:28.566973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 13:56:28.566973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [parameters: {'id': '17afb59c607244c18cee8f98e39608bb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 13:56:28.566973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 13:56:28.566973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:28.566973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:28.566973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:28.566973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:28.566973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 18 13:56:28.570564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 18 13:56:28.570564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 13:56:28.570564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 13:56:28.570564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 18 13:56:28.570564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:28.570564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:28.570564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:28.570564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:28.570564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:28.570564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:28.570564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:28.570564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:28.570564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:28.570564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:28.570564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:28.570564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:28.570564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:28.573507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:28.573507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 18 13:56:28.573507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 18 13:56:28.573507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:28.573507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:28.573507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 18 13:56:28.573507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 18 13:56:28.573507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 17afb59c607244c18cee8f98e39608bb. Jul 18 13:56:28.573507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:28.573507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 462/939] 10.222.0.94 () {60 vars in 1140 bytes} [Mon Jul 18 13:56:28 2022] POST /identity/v3/projects => generated 262 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:28.573507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bcde71-85cb-4ac3-9588-6b96a8bb9a58 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:28.618000 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff3c2c-d60a-4952-b01d-7e98686d82e9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:28.643270 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:28.654839 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad066c0-d612-45d6-bf17-0318d7789405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:28.655515 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad066c0-d612-45d6-bf17-0318d7789405 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:28.655803 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad066c0-d612-45d6-bf17-0318d7789405 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:28.656122 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad066c0-d612-45d6-bf17-0318d7789405 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:28.656859 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad066c0-d612-45d6-bf17-0318d7789405 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:28.657174 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad066c0-d612-45d6-bf17-0318d7789405 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:28.658581 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad066c0-d612-45d6-bf17-0318d7789405 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:28.675435 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 478/940] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:28.675435 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad066c0-d612-45d6-bf17-0318d7789405 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:28.684063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bcde71-85cb-4ac3-9588-6b96a8bb9a58 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:28.720896 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226ebf6-8bb6-46e0-afa4-b1930c6f4f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:28.721642 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226ebf6-8bb6-46e0-afa4-b1930c6f4f4d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:28.721944 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226ebf6-8bb6-46e0-afa4-b1930c6f4f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:28.722233 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226ebf6-8bb6-46e0-afa4-b1930c6f4f4d None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:28.723567 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226ebf6-8bb6-46e0-afa4-b1930c6f4f4d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:28.723892 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226ebf6-8bb6-46e0-afa4-b1930c6f4f4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:28.726500 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226ebf6-8bb6-46e0-afa4-b1930c6f4f4d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:28.741107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 463/941] 10.222.0.94 () {56 vars in 1122 bytes} [Mon Jul 18 13:56:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:28.741107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226ebf6-8bb6-46e0-afa4-b1930c6f4f4d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:28.775072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad066c0-d612-45d6-bf17-0318d7789405 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:28.790034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:28.813575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc898a-ab65-435a-b0bb-07b47c15ec1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:28.814677 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc898a-ab65-435a-b0bb-07b47c15ec1e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:28.814977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc898a-ab65-435a-b0bb-07b47c15ec1e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:28.815260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc898a-ab65-435a-b0bb-07b47c15ec1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:28.822736 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc898a-ab65-435a-b0bb-07b47c15ec1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:28.824969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc898a-ab65-435a-b0bb-07b47c15ec1e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:28.840693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-eedc898a-ab65-435a-b0bb-07b47c15ec1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:28.840693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:28.840693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:28.840693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:28.840693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:28.840693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:28.840693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:28.840693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:28.840693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:28.840693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:28.840693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:28.840693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:28.840693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:56:28.840693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:56:28.840693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:56:28.841600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:56:28.841600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:28.841600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:28.841600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:28.841600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:28.841600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:28.841600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:28.841600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:28.841600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:28.841600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:28.841600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:28.841600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:28.841600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:28.841600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:28.841600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:28.841600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:28.841600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:28.842496 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:56:28.842496 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:56:28.842496 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:56:28.842496 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:56:28.842496 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:56:28.842496 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:56:28.842496 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:28.842496 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:28.842496 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 479/942] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:56:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:28.843013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc898a-ab65-435a-b0bb-07b47c15ec1e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:28.857443 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226ebf6-8bb6-46e0-afa4-b1930c6f4f4d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:28.898167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a029bc-4d33-484f-a295-3adca14dd690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:28.898964 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a029bc-4d33-484f-a295-3adca14dd690 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:28.899257 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a029bc-4d33-484f-a295-3adca14dd690 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:28.899534 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a029bc-4d33-484f-a295-3adca14dd690 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:28.902062 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:28.908484 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a029bc-4d33-484f-a295-3adca14dd690 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:28.910707 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a029bc-4d33-484f-a295-3adca14dd690 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:28.924952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-b8a029bc-4d33-484f-a295-3adca14dd690 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:28.924952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:28.924952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:28.924952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:28.924952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:28.924952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:28.924952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:28.924952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:28.924952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:28.924952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:28.924952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:28.924952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:28.924952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:56:28.924952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:56:28.924952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:56:28.926121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:56:28.926121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:28.926121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:28.926121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:28.926121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:28.926121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:28.926121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:28.926121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:28.926121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:28.926121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:28.926121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:28.926121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:28.926121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:28.926121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:28.926121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:28.926121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:28.926121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:28.927034 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:56:28.927034 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:56:28.927034 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:56:28.927034 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:56:28.927034 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:56:28.927034 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:56:28.927034 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:28.927034 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:28.927034 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 464/943] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:56:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:28.927034 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a029bc-4d33-484f-a295-3adca14dd690 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:28.960597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc898a-ab65-435a-b0bb-07b47c15ec1e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:28.996630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf6430-778e-4dcb-8671-a47c60fe56b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:28.997402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf6430-778e-4dcb-8671-a47c60fe56b4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:28.997684 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf6430-778e-4dcb-8671-a47c60fe56b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:28.997960 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf6430-778e-4dcb-8671-a47c60fe56b4 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:28.998664 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf6430-778e-4dcb-8671-a47c60fe56b4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:28.998972 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf6430-778e-4dcb-8671-a47c60fe56b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:29.000359 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf6430-778e-4dcb-8671-a47c60fe56b4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:29.005274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 480/944] 10.222.0.94 () {56 vars in 1122 bytes} [Mon Jul 18 13:56:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:29.006049 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf6430-778e-4dcb-8671-a47c60fe56b4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:29.044578 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:29.044954 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a029bc-4d33-484f-a295-3adca14dd690 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:29.088131 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956fb4e-ea04-45ba-976b-4870a75137e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:29.088805 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956fb4e-ea04-45ba-976b-4870a75137e0 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:29.089090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956fb4e-ea04-45ba-976b-4870a75137e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:29.089365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956fb4e-ea04-45ba-976b-4870a75137e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:29.090121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a956fb4e-ea04-45ba-976b-4870a75137e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:29.090486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a956fb4e-ea04-45ba-976b-4870a75137e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:29.092421 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a956fb4e-ea04-45ba-976b-4870a75137e0 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:29.119977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 465/945] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Jul 18 13:56:28 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:29.119977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956fb4e-ea04-45ba-976b-4870a75137e0 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:29.120293 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf6430-778e-4dcb-8671-a47c60fe56b4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:29.163355 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297548e-d887-4140-9ca9-1d482bca59b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:29.164294 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297548e-d887-4140-9ca9-1d482bca59b7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:29.164942 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297548e-d887-4140-9ca9-1d482bca59b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:29.165439 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297548e-d887-4140-9ca9-1d482bca59b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:29.166453 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297548e-d887-4140-9ca9-1d482bca59b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:29.166949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297548e-d887-4140-9ca9-1d482bca59b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:29.169271 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297548e-d887-4140-9ca9-1d482bca59b7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:29.187872 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 481/946] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:56:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:29.195656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297548e-d887-4140-9ca9-1d482bca59b7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:29.206263 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:29.263291 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956fb4e-ea04-45ba-976b-4870a75137e0 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:29.301979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4131e003-4351-41d8-8241-dff9f27034d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:29.302655 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131e003-4351-41d8-8241-dff9f27034d1 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:29.302948 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131e003-4351-41d8-8241-dff9f27034d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:29.303262 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131e003-4351-41d8-8241-dff9f27034d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:29.304556 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4131e003-4351-41d8-8241-dff9f27034d1 None admin] RBAC: Adding query filter params (interface=public, service_id=d65d96de56554680a009f71a30e44c6a, region_id=RegionOne) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:29.304878 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4131e003-4351-41d8-8241-dff9f27034d1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:29.306742 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4131e003-4351-41d8-8241-dff9f27034d1 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:29.312566 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:29.313333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 466/947] 10.222.0.94 () {56 vars in 1251 bytes} [Mon Jul 18 13:56:29 2022] GET /identity/v3/endpoints?service_id=d65d96de56554680a009f71a30e44c6a&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:29.319131 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4131e003-4351-41d8-8241-dff9f27034d1 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:29.322241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297548e-d887-4140-9ca9-1d482bca59b7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:29.369415 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f166a4-956d-4bb5-bd2f-fe7d9c205bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:29.370838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f166a4-956d-4bb5-bd2f-fe7d9c205bbb None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:29.371124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f166a4-956d-4bb5-bd2f-fe7d9c205bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:29.371404 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f166a4-956d-4bb5-bd2f-fe7d9c205bbb None admin] PATH_INFO: `/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/94d824bffc2c4af299c537056bc21734/roles/039f54fd5a574af19dad80e091f25108` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:29.373987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f166a4-956d-4bb5-bd2f-fe7d9c205bbb None admin] RBAC: Authorizing `identity:create_grant(project_id=ec94946acf91486ba3f5b4fe834b8076, user_id=94d824bffc2c4af299c537056bc21734, role_id=039f54fd5a574af19dad80e091f25108)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:29.375758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f166a4-956d-4bb5-bd2f-fe7d9c205bbb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:29.390530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 482/948] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:56:29 2022] PUT /identity/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/94d824bffc2c4af299c537056bc21734/roles/039f54fd5a574af19dad80e091f25108 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:56:29.392186 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b1347-81e8-478c-a3ff-5e3334cd3984 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:29.392704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b1347-81e8-478c-a3ff-5e3334cd3984 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:29.393190 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b1347-81e8-478c-a3ff-5e3334cd3984 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:29.400675 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 483/949] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:29.405207 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5b838-408e-4ac4-99a3-3387d33dead9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:29.405866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5b838-408e-4ac4-99a3-3387d33dead9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:29.406367 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5b838-408e-4ac4-99a3-3387d33dead9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:29.424405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c4a5b838-408e-4ac4-99a3-3387d33dead9 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:29.426007 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c4a5b838-408e-4ac4-99a3-3387d33dead9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:29.441870 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4131e003-4351-41d8-8241-dff9f27034d1 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:29.464510 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:29.486109 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4573a6-5fb8-46a5-b511-4a73fa2d2b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:29.486799 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4573a6-5fb8-46a5-b511-4a73fa2d2b43 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:29.487060 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4573a6-5fb8-46a5-b511-4a73fa2d2b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:29.487347 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4573a6-5fb8-46a5-b511-4a73fa2d2b43 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:29.488051 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4573a6-5fb8-46a5-b511-4a73fa2d2b43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:29.489504 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4573a6-5fb8-46a5-b511-4a73fa2d2b43 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:29.496877 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 467/950] 10.222.0.94 () {56 vars in 1087 bytes} [Mon Jul 18 13:56:29 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 13:56:29.533790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 484/951] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 13:56:29.557734 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4573a6-5fb8-46a5-b511-4a73fa2d2b43 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:29.678396 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4573a6-5fb8-46a5-b511-4a73fa2d2b43 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:29.715294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae7bd8-0230-4dfb-820f-b447774fb387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:29.716129 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae7bd8-0230-4dfb-820f-b447774fb387 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:29.716425 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae7bd8-0230-4dfb-820f-b447774fb387 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:29.716708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae7bd8-0230-4dfb-820f-b447774fb387 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:29.717375 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae7bd8-0230-4dfb-820f-b447774fb387 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:29.718751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae7bd8-0230-4dfb-820f-b447774fb387 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:29.731302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-2cae7bd8-0230-4dfb-820f-b447774fb387 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 13:56:29.731302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:29.731302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:56:29.731302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:56:29.731302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:56:29.731302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:56:29.731302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:56:29.731302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:29.731302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 13:56:29.731302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:29.731302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:29.731302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:29.731302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:29.731302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:29.731302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:29.731302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:29.731302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:29.732650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:29.732650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:29.732650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:29.732650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:29.732650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:29.732650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:29.732650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:56:29.732650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:56:29.732650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:56:29.732650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:56:29.732650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:29.732650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:29.732650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:29.732650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:29.732650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:29.732650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:29.733766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:29.733766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:29.733766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:29.733766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:29.733766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:29.733766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:29.733766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:29.733766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:29.733766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:29.733766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:29.733766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:56:29.733766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:29.733766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 13:56:29.733766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:29.733766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 468/952] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:56:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:29.738485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5b838-408e-4ac4-99a3-3387d33dead9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:29.826624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:29.843409 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5b838-408e-4ac4-99a3-3387d33dead9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:29.880574 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a662d3-d6e7-4dd7-bfc9-0166f7d7a59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:29.881529 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a662d3-d6e7-4dd7-bfc9-0166f7d7a59f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:29.881893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a662d3-d6e7-4dd7-bfc9-0166f7d7a59f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:29.882245 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a662d3-d6e7-4dd7-bfc9-0166f7d7a59f None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:29.883034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a662d3-d6e7-4dd7-bfc9-0166f7d7a59f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:29.883421 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a662d3-d6e7-4dd7-bfc9-0166f7d7a59f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:29.884892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a662d3-d6e7-4dd7-bfc9-0166f7d7a59f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:29.890355 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 485/953] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Jul 18 13:56:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:29.897613 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae7bd8-0230-4dfb-820f-b447774fb387 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:29.947550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:30.008859 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae7bd8-0230-4dfb-820f-b447774fb387 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:30.045275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a3d68-a161-402e-ade0-7869ceecd918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:30.046187 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a3d68-a161-402e-ade0-7869ceecd918 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:30.046658 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a3d68-a161-402e-ade0-7869ceecd918 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:30.047084 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a3d68-a161-402e-ade0-7869ceecd918 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:30.047931 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a3d68-a161-402e-ade0-7869ceecd918 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:30.048425 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a3d68-a161-402e-ade0-7869ceecd918 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:30.049939 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a3d68-a161-402e-ade0-7869ceecd918 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:30.055725 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 469/954] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Jul 18 13:56:29 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:31.241227 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9997fc-a483-4888-acfb-a5e5b3fe870f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:31.241715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9997fc-a483-4888-acfb-a5e5b3fe870f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:31.241715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9997fc-a483-4888-acfb-a5e5b3fe870f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:31.243337 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 486/955] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:31.248673 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113c3af-a118-4713-9663-867b52690736 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:31.249089 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113c3af-a118-4713-9663-867b52690736 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:31.249266 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113c3af-a118-4713-9663-867b52690736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:31.267028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f113c3af-a118-4713-9663-867b52690736 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:31.268596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f113c3af-a118-4713-9663-867b52690736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:31.339393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 470/956] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:31.364141 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9997fc-a483-4888-acfb-a5e5b3fe870f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:31.466053 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9997fc-a483-4888-acfb-a5e5b3fe870f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:31.503689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32bb89d-0ae3-4a44-bb83-a4766ef4ce4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:31.504320 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bb89d-0ae3-4a44-bb83-a4766ef4ce4e None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:31.504604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bb89d-0ae3-4a44-bb83-a4766ef4ce4e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:31.504877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bb89d-0ae3-4a44-bb83-a4766ef4ce4e None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:31.505579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32bb89d-0ae3-4a44-bb83-a4766ef4ce4e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:31.507046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32bb89d-0ae3-4a44-bb83-a4766ef4ce4e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:31.508375 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: WARNING keystone.server.flask.common [None req-f32bb89d-0ae3-4a44-bb83-a4766ef4ce4e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 18 13:56:31.515107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21ce7d-3f35-4f11-bcd4-1cf96785b92d None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:31.515403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21ce7d-3f35-4f11-bcd4-1cf96785b92d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:31.515690 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21ce7d-3f35-4f11-bcd4-1cf96785b92d None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:31.518329 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 471/957] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:31.520803 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.sql.core [None req-f32bb89d-0ae3-4a44-bb83-a4766ef4ce4e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 13:56:31.520803 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 13:56:31.520803 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [parameters: {'user_id': 'b758c65725404cb8b3831e185f1017a8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 13:56:31.520803 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80160) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 13:56:31.523543 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12c5c2-f0ac-47b1-ae01-9ec489996b70 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:31.523836 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12c5c2-f0ac-47b1-ae01-9ec489996b70 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:31.524159 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12c5c2-f0ac-47b1-ae01-9ec489996b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:31.531297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f32bb89d-0ae3-4a44-bb83-a4766ef4ce4e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 13:56:31.531297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:31.531297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 13:56:31.531297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 13:56:31.531297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 13:56:31.531297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 13:56:31.531297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 13:56:31.531297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 13:56:31.531297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 13:56:31.531297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application conn.query(q) Jul 18 13:56:31.531297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 13:56:31.531297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 13:56:31.531297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 13:56:31.531297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result.read() Jul 18 13:56:31.531297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 13:56:31.532457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 13:56:31.532457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 13:56:31.532457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 13:56:31.532457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 13:56:31.532457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 13:56:31.532457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 13:56:31.532457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 13:56:31.532457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 13:56:31.532457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:31.532457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 13:56:31.532457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:31.532457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:31.532457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 13:56:31.532457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 13:56:31.532457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 18 13:56:31.532457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 18 13:56:31.532457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 13:56:31.532457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application next(self.gen) Jul 18 13:56:31.533699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 13:56:31.533699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application yield resource Jul 18 13:56:31.533699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 13:56:31.533699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application next(self.gen) Jul 18 13:56:31.533699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 13:56:31.533699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 13:56:31.533699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 13:56:31.533699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 13:56:31.533699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 13:56:31.533699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise self.value Jul 18 13:56:31.533699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 13:56:31.533699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 13:56:31.533699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 13:56:31.533699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application session.commit() Jul 18 13:56:31.533699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 13:56:31.533699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 13:56:31.533699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 13:56:31.534704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 13:56:31.534704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 13:56:31.534704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.session.flush() Jul 18 13:56:31.534704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 13:56:31.534704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 13:56:31.534704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 13:56:31.534704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 13:56:31.534704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 13:56:31.534704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application compat.raise_( Jul 18 13:56:31.534704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 13:56:31.534704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception Jul 18 13:56:31.534704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 13:56:31.534704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 13:56:31.534704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 13:56:31.534704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 13:56:31.534704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 13:56:31.534704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 13:56:31.536165 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 13:56:31.536165 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 13:56:31.536165 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 18 13:56:31.536165 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 18 13:56:31.536165 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 13:56:31.536165 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 13:56:31.536165 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 13:56:31.536165 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 13:56:31.536165 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 13:56:31.536165 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 13:56:31.536165 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 13:56:31.536165 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 13:56:31.536165 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 13:56:31.536165 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 13:56:31.536165 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 13:56:31.536165 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception Jul 18 13:56:31.537115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 13:56:31.537115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 13:56:31.537115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 13:56:31.537115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 13:56:31.537115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 13:56:31.537115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 13:56:31.537115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 13:56:31.537115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application conn.query(q) Jul 18 13:56:31.537115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 13:56:31.537115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 13:56:31.537115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 13:56:31.537115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result.read() Jul 18 13:56:31.537115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 13:56:31.537115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 13:56:31.537115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 13:56:31.537115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 13:56:31.538525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 13:56:31.538525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 13:56:31.538525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 13:56:31.538525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 13:56:31.538525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 13:56:31.538525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 13:56:31.538525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b758c65725404cb8b3831e185f1017a8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 13:56:31.538525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 13:56:31.538525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:31.538525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:31.538525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:31.538525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:31.538525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:31.538525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:31.538525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:31.539767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:31.539767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:31.539767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:31.539767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:31.539767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:31.539767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:31.539767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:31.539767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 18 13:56:31.539767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 18 13:56:31.539767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:31.539767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:31.539767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:56:31.539767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:31.539767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:56:31.539767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:31.539767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 18 13:56:31.539767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 18 13:56:31.540806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 18 13:56:31.540806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 18 13:56:31.540806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 13:56:31.540806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 13:56:31.540806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 13:56:31.540806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:31.542976 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 487/958] 10.222.0.94 () {60 vars in 1131 bytes} [Mon Jul 18 13:56:31 2022] POST /identity/v3/users => generated 160 bytes in 179 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:31.543175 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-2c12c5c2-f0ac-47b1-ae01-9ec489996b70 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:31.548019 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2c12c5c2-f0ac-47b1-ae01-9ec489996b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:31.549203 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32bb89d-0ae3-4a44-bb83-a4766ef4ce4e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:31.566379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:31.593721 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:31.643046 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 472/959] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 13:56:31.644456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d72da-bc6c-4c15-b0fd-37c93ef45456 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:31.644755 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d72da-bc6c-4c15-b0fd-37c93ef45456 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:31.645039 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d72da-bc6c-4c15-b0fd-37c93ef45456 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:31.646469 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 473/960] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:31.652340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ea28f-3a2f-45ad-9f45-4fce10f7c81f None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:31.652489 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ea28f-3a2f-45ad-9f45-4fce10f7c81f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:31.652711 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ea28f-3a2f-45ad-9f45-4fce10f7c81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:31.670343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-975ea28f-3a2f-45ad-9f45-4fce10f7c81f None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:31.671518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-975ea28f-3a2f-45ad-9f45-4fce10f7c81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:31.674071 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32bb89d-0ae3-4a44-bb83-a4766ef4ce4e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:31.712296 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82a085-ad9a-4a7a-b3bb-daae2869d6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:31.713251 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82a085-ad9a-4a7a-b3bb-daae2869d6f0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:31.713544 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82a085-ad9a-4a7a-b3bb-daae2869d6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:31.713826 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82a085-ad9a-4a7a-b3bb-daae2869d6f0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:31.722240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82a085-ad9a-4a7a-b3bb-daae2869d6f0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:31.724006 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82a085-ad9a-4a7a-b3bb-daae2869d6f0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:31.744602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 474/961] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:31.744602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ea28f-3a2f-45ad-9f45-4fce10f7c81f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:31.744885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-8d82a085-ad9a-4a7a-b3bb-daae2869d6f0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 13:56:31.744885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:31.744885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:31.744885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:31.744885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:31.744885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:31.744885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:31.744885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:31.744885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:31.744885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:31.744885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:31.744885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:31.744885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:56:31.744885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:56:31.744885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:56:31.744885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:56:31.745877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:31.745877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:31.745877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:56:31.745877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:31.745877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:56:31.745877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:31.745877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:31.745877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:31.745877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:31.745877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:31.745877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:31.745877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:31.745877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:31.745877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:31.745877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:31.745877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:31.745877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:31.745877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:31.746949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:31.746949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:31.746949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:56:31.746949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:56:31.746949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:56:31.746949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:56:31.746949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:56:31.746949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:56:31.746949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 13:56:31.746949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:31.746949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 488/962] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:56:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:31.746949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82a085-ad9a-4a7a-b3bb-daae2869d6f0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:31.825490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:31.859174 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82a085-ad9a-4a7a-b3bb-daae2869d6f0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:31.866391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:31.870284 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ea28f-3a2f-45ad-9f45-4fce10f7c81f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:31.896007 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75ea31-12a8-44a2-898b-c334aa71fa98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:31.897101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75ea31-12a8-44a2-898b-c334aa71fa98 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:31.897394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75ea31-12a8-44a2-898b-c334aa71fa98 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:31.898090 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75ea31-12a8-44a2-898b-c334aa71fa98 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:31.899187 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75ea31-12a8-44a2-898b-c334aa71fa98 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:31.899504 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75ea31-12a8-44a2-898b-c334aa71fa98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:31.901911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75ea31-12a8-44a2-898b-c334aa71fa98 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:31.905420 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd09ef-9800-40d4-a949-21cd1a09cc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:31.906231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd09ef-9800-40d4-a949-21cd1a09cc3f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:31.906520 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd09ef-9800-40d4-a949-21cd1a09cc3f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:31.906792 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd09ef-9800-40d4-a949-21cd1a09cc3f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:31.917992 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd09ef-9800-40d4-a949-21cd1a09cc3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:56:31.926114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-5cdd09ef-9800-40d4-a949-21cd1a09cc3f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 13:56:31.926114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:31.926114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:31.926114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:31.926114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:31.926114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:31.926114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:31.926114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:31.926114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:31.926114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:31.926114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:31.926114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:31.926114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:56:31.926114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:56:31.926114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:56:31.926114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:56:31.927075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:56:31.927075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:56:31.927075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:56:31.927075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:56:31.927075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:31.927075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:31.927075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:31.927075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:31.927075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:31.927075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:31.927075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:31.927075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:31.927075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:31.927075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:31.927075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:31.927075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:31.932910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:31.932910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:31.932910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:31.932910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:31.932910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:56:31.932910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:56:31.932910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:56:31.932910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:56:31.932910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:56:31.932910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:56:31.932910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 13:56:31.932910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:31.932910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 475/963] 10.222.0.94 () {56 vars in 1120 bytes} [Mon Jul 18 13:56:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:31.932910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd09ef-9800-40d4-a949-21cd1a09cc3f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:31.952334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 489/964] 10.222.0.94 () {56 vars in 1101 bytes} [Mon Jul 18 13:56:31 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:31.954649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75ea31-12a8-44a2-898b-c334aa71fa98 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:32.075108 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd09ef-9800-40d4-a949-21cd1a09cc3f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:32.104385 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75ea31-12a8-44a2-898b-c334aa71fa98 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:32.128075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9e2b9-927f-441b-9827-07aeb1d7f28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:32.128903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9e2b9-927f-441b-9827-07aeb1d7f28e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:32.129199 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9e2b9-927f-441b-9827-07aeb1d7f28e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:32.129480 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9e2b9-927f-441b-9827-07aeb1d7f28e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:32.130641 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9e2b9-927f-441b-9827-07aeb1d7f28e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:32.132609 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9e2b9-927f-441b-9827-07aeb1d7f28e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:32.142702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:32.145533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-32a9e2b9-927f-441b-9827-07aeb1d7f28e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 13:56:32.145533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:32.145533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:56:32.145533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:56:32.145533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:56:32.145533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:56:32.145533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:56:32.145533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:32.145533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 13:56:32.145533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:32.145533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:32.145533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:32.145533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:32.145533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:32.145533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:32.145533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:32.145533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:32.147790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:32.147790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:32.147790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:32.147790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:32.147790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:32.147790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:32.147790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:56:32.147790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:56:32.147790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:56:32.147790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:56:32.147790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:32.147790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:32.147790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:32.147790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:32.147790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:32.147790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:32.148713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:32.148713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:32.148713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:32.148713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:32.148713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:32.148713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:32.148713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:32.148713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:32.148713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:32.148713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:32.148713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:56:32.148713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:32.148713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 13:56:32.148713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:32.148713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 476/965] 10.222.0.94 () {56 vars in 1126 bytes} [Mon Jul 18 13:56:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:32.150019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab84e4-d5df-4dad-95cb-3b5bfccc5df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:32.150019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab84e4-d5df-4dad-95cb-3b5bfccc5df1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:32.150019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab84e4-d5df-4dad-95cb-3b5bfccc5df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:32.150019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab84e4-d5df-4dad-95cb-3b5bfccc5df1 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:32.150372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab84e4-d5df-4dad-95cb-3b5bfccc5df1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:32.150372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab84e4-d5df-4dad-95cb-3b5bfccc5df1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:32.151419 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab84e4-d5df-4dad-95cb-3b5bfccc5df1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:32.152987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52b8aa-b89b-44b2-b575-83cccf015d58 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:32.153264 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52b8aa-b89b-44b2-b575-83cccf015d58 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:32.153545 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52b8aa-b89b-44b2-b575-83cccf015d58 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:32.155004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 477/966] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:32.157049 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52b8aa-b89b-44b2-b575-83cccf015d58 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:32.162587 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 490/967] 10.222.0.94 () {56 vars in 1115 bytes} [Mon Jul 18 13:56:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:32.164136 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f83b28-0891-415e-ae4a-07f1107e1a46 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:32.164437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f83b28-0891-415e-ae4a-07f1107e1a46 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:32.164720 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f83b28-0891-415e-ae4a-07f1107e1a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:32.184454 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-23f83b28-0891-415e-ae4a-07f1107e1a46 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:32.186240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-23f83b28-0891-415e-ae4a-07f1107e1a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:32.191820 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:32.257233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 491/968] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:32.258669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f83b28-0891-415e-ae4a-07f1107e1a46 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:32.274574 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52b8aa-b89b-44b2-b575-83cccf015d58 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:32.308686 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb165862-ac59-44c4-909d-6849eeafdd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:32.309353 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb165862-ac59-44c4-909d-6849eeafdd5b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:32.309640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb165862-ac59-44c4-909d-6849eeafdd5b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:32.309926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb165862-ac59-44c4-909d-6849eeafdd5b None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:32.310629 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb165862-ac59-44c4-909d-6849eeafdd5b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:32.310933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb165862-ac59-44c4-909d-6849eeafdd5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:32.312334 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb165862-ac59-44c4-909d-6849eeafdd5b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:32.317150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 478/969] 10.222.0.94 () {56 vars in 1122 bytes} [Mon Jul 18 13:56:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:32.317726 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb165862-ac59-44c4-909d-6849eeafdd5b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:32.359406 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f83b28-0891-415e-ae4a-07f1107e1a46 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:32.382313 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:32.394183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe9856-0f72-4795-a565-4f30aa78fb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:32.394972 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe9856-0f72-4795-a565-4f30aa78fb0d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:32.395262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe9856-0f72-4795-a565-4f30aa78fb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:32.395540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe9856-0f72-4795-a565-4f30aa78fb0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:32.406244 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:32.416980 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe9856-0f72-4795-a565-4f30aa78fb0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:32.418432 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe9856-0f72-4795-a565-4f30aa78fb0d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:32.419241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb165862-ac59-44c4-909d-6849eeafdd5b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:32.436651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-edfe9856-0f72-4795-a565-4f30aa78fb0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:32.436651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:32.436651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:32.436651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:32.436651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:32.436651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:32.436651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:32.436651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:32.436651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:32.436651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:32.436651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:32.436651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:32.436651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:32.436651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:32.436651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:32.437858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:32.437858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:32.437858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:32.437858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:32.437858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:32.437858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:32.437858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:32.437858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:32.437858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:32.437858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:32.437858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:32.437858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:32.437858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:32.437858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:32.437858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:32.437858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:32.437858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:32.438791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:32.438791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:32.438791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:32.438791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:32.438791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 492/970] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:32.438791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe9856-0f72-4795-a565-4f30aa78fb0d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:32.458198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d3d8f-7418-4b2d-8252-cef00938da36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:32.458874 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d3d8f-7418-4b2d-8252-cef00938da36 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:32.459166 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d3d8f-7418-4b2d-8252-cef00938da36 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:32.459515 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d3d8f-7418-4b2d-8252-cef00938da36 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:32.460294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d3d8f-7418-4b2d-8252-cef00938da36 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:32.461672 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d3d8f-7418-4b2d-8252-cef00938da36 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:32.474505 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 479/971] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Jul 18 13:56:32 2022] POST /identity/v3/services => generated 242 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:56:32.475400 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d3d8f-7418-4b2d-8252-cef00938da36 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:32.563448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe9856-0f72-4795-a565-4f30aa78fb0d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:32.597611 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d3d8f-7418-4b2d-8252-cef00938da36 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:32.601706 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747079e-56a5-4f69-ba31-42fd93535af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:32.602382 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747079e-56a5-4f69-ba31-42fd93535af1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:32.602667 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747079e-56a5-4f69-ba31-42fd93535af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:32.602944 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747079e-56a5-4f69-ba31-42fd93535af1 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:32.603689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747079e-56a5-4f69-ba31-42fd93535af1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:32.604039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747079e-56a5-4f69-ba31-42fd93535af1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:32.605438 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747079e-56a5-4f69-ba31-42fd93535af1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:32.611449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 493/972] 10.222.0.94 () {56 vars in 1122 bytes} [Mon Jul 18 13:56:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:32.617354 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747079e-56a5-4f69-ba31-42fd93535af1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:32.648765 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76757f-bf03-4d4f-8806-0a5ec36c73ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:32.649715 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76757f-bf03-4d4f-8806-0a5ec36c73ee None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:32.649715 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76757f-bf03-4d4f-8806-0a5ec36c73ee None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:32.649887 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76757f-bf03-4d4f-8806-0a5ec36c73ee None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:32.650593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76757f-bf03-4d4f-8806-0a5ec36c73ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:32.650904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76757f-bf03-4d4f-8806-0a5ec36c73ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:32.652309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76757f-bf03-4d4f-8806-0a5ec36c73ee None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:32.665685 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 480/973] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:32.666012 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:32.671322 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76757f-bf03-4d4f-8806-0a5ec36c73ee None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:32.675463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:32.734014 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747079e-56a5-4f69-ba31-42fd93535af1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:32.774639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9789fe4-6682-4196-8fe6-91a06ff96500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:32.775395 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9789fe4-6682-4196-8fe6-91a06ff96500 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:32.775678 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9789fe4-6682-4196-8fe6-91a06ff96500 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:32.775966 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9789fe4-6682-4196-8fe6-91a06ff96500 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:32.776708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9789fe4-6682-4196-8fe6-91a06ff96500 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:32.776826 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76757f-bf03-4d4f-8806-0a5ec36c73ee None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:32.778075 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9789fe4-6682-4196-8fe6-91a06ff96500 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:32.807986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 494/974] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Jul 18 13:56:32 2022] POST /identity/v3/endpoints => generated 349 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 13:56:32.832677 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9a24d-1f6b-4d01-86b1-e96f5f8c0939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:32.833678 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9a24d-1f6b-4d01-86b1-e96f5f8c0939 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:32.834067 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9a24d-1f6b-4d01-86b1-e96f5f8c0939 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:32.834440 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9a24d-1f6b-4d01-86b1-e96f5f8c0939 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:32.843009 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9a24d-1f6b-4d01-86b1-e96f5f8c0939 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:32.844544 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9a24d-1f6b-4d01-86b1-e96f5f8c0939 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:32.861803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-dbd9a24d-1f6b-4d01-86b1-e96f5f8c0939 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 13:56:32.861803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:32.861803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:32.861803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:32.861803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:32.861803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:32.861803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:32.861803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:32.861803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:32.861803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:32.861803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:32.861803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:32.861803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:56:32.861803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:56:32.861803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:56:32.861803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:56:32.862795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:32.862795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:32.862795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:56:32.862795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:32.862795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:56:32.862795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:32.862795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:32.862795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:32.862795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:32.862795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:32.862795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:32.862795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:32.862795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:32.862795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:32.862795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:32.862795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:32.862795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:32.862795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:32.863755 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:32.863755 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:32.863755 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:56:32.863755 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:56:32.863755 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:56:32.863755 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:56:32.863755 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:56:32.863755 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:56:32.863755 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 13:56:32.863755 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:32.863755 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 481/975] 10.222.0.94 () {56 vars in 1117 bytes} [Mon Jul 18 13:56:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:32.871718 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9789fe4-6682-4196-8fe6-91a06ff96500 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:32.985105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:32.993340 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9789fe4-6682-4196-8fe6-91a06ff96500 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:33.029648 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0cb28-c32a-4135-88ba-dcdea8c4a32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:33.030625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0cb28-c32a-4135-88ba-dcdea8c4a32a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:33.031115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0cb28-c32a-4135-88ba-dcdea8c4a32a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:33.031587 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0cb28-c32a-4135-88ba-dcdea8c4a32a None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:33.032621 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de0cb28-c32a-4135-88ba-dcdea8c4a32a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:33.033112 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de0cb28-c32a-4135-88ba-dcdea8c4a32a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:33.034700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de0cb28-c32a-4135-88ba-dcdea8c4a32a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:33.052807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 495/976] 10.222.0.94 () {56 vars in 1149 bytes} [Mon Jul 18 13:56:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:33.058644 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9a24d-1f6b-4d01-86b1-e96f5f8c0939 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:33.128534 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:33.171376 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9a24d-1f6b-4d01-86b1-e96f5f8c0939 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:33.206982 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e0d18-7d34-4450-a699-334368f0aa8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:33.207937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e0d18-7d34-4450-a699-334368f0aa8c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:33.208459 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e0d18-7d34-4450-a699-334368f0aa8c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:33.208920 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e0d18-7d34-4450-a699-334368f0aa8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:33.276877 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e0d18-7d34-4450-a699-334368f0aa8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:33.278999 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e0d18-7d34-4450-a699-334368f0aa8c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:33.335898 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:33.378770 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 482/977] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:56:33 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 13:56:33.384309 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0cb28-c32a-4135-88ba-dcdea8c4a32a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:33.482832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0cb28-c32a-4135-88ba-dcdea8c4a32a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:33.516771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8943773-baa0-4766-af3c-0a41a5b3ff36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:33.517571 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8943773-baa0-4766-af3c-0a41a5b3ff36 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:33.517869 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8943773-baa0-4766-af3c-0a41a5b3ff36 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:33.518154 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8943773-baa0-4766-af3c-0a41a5b3ff36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:33.528883 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8943773-baa0-4766-af3c-0a41a5b3ff36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:33.530328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8943773-baa0-4766-af3c-0a41a5b3ff36 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:33.548275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-d8943773-baa0-4766-af3c-0a41a5b3ff36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:33.548275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:33.548275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:33.548275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:33.548275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:33.548275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:33.548275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:33.548275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:33.548275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:33.548275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:33.548275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:33.548275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:33.548275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:33.548275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:33.548275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:33.549198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:33.549198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:33.549198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:33.549198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:33.549198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:33.549198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:33.549198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:33.549198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:33.549198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:33.549198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:33.549198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:33.549198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:33.549198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:33.549198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:33.549198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:33.549198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:33.549198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:33.555129 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:33.555129 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:33.555129 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:33.555129 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:33.555129 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 496/978] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:33.556548 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e0d18-7d34-4450-a699-334368f0aa8c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:33.659903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e0d18-7d34-4450-a699-334368f0aa8c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:33.694051 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4f1d2-b056-4a05-9621-9e3d666cca02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:33.694714 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4f1d2-b056-4a05-9621-9e3d666cca02 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:33.695002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4f1d2-b056-4a05-9621-9e3d666cca02 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:33.695284 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4f1d2-b056-4a05-9621-9e3d666cca02 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:33.696024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4f1d2-b056-4a05-9621-9e3d666cca02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:33.696338 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4f1d2-b056-4a05-9621-9e3d666cca02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:33.697704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4f1d2-b056-4a05-9621-9e3d666cca02 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:33.714168 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 483/979] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:33.715464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8943773-baa0-4766-af3c-0a41a5b3ff36 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:33.720222 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:33.820191 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8943773-baa0-4766-af3c-0a41a5b3ff36 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:33.854610 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6d5d8-0c55-44f4-9a3a-5dbf5e2a1c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:33.855409 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6d5d8-0c55-44f4-9a3a-5dbf5e2a1c12 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:33.855698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6d5d8-0c55-44f4-9a3a-5dbf5e2a1c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:33.856000 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6d5d8-0c55-44f4-9a3a-5dbf5e2a1c12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:33.862663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6d5d8-0c55-44f4-9a3a-5dbf5e2a1c12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:33.864172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6d5d8-0c55-44f4-9a3a-5dbf5e2a1c12 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:33.879093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5ae6d5d8-0c55-44f4-9a3a-5dbf5e2a1c12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:33.879093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:33.879093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:33.879093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:33.879093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:33.879093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:33.879093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:33.879093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:33.879093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:33.879093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:33.879093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:33.879093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:33.879093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:56:33.879093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:56:33.879093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:56:33.880021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:56:33.880021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:33.880021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:33.880021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:33.880021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:33.880021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:33.880021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:33.880021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:33.880021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:33.880021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:33.880021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:33.880021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:33.880021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:33.880021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:33.880021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:33.880021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:33.880021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:33.880933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:56:33.880933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:56:33.880933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:56:33.880933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:56:33.880933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:56:33.880933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:56:33.880933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:33.880933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:33.880933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 497/980] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:56:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:33.886070 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4f1d2-b056-4a05-9621-9e3d666cca02 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:33.909652 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:33.997130 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4f1d2-b056-4a05-9621-9e3d666cca02 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:34.031510 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17fad0-2e50-4c52-9efc-b419475735b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:34.032196 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17fad0-2e50-4c52-9efc-b419475735b0 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:34.032482 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17fad0-2e50-4c52-9efc-b419475735b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:34.032759 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17fad0-2e50-4c52-9efc-b419475735b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:34.033538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17fad0-2e50-4c52-9efc-b419475735b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:34.033854 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17fad0-2e50-4c52-9efc-b419475735b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:34.035279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf17fad0-2e50-4c52-9efc-b419475735b0 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:34.048131 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 484/981] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:56:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:34.054363 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6d5d8-0c55-44f4-9a3a-5dbf5e2a1c12 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:34.116824 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:34.156557 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6d5d8-0c55-44f4-9a3a-5dbf5e2a1c12 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:34.189146 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c577f9-138f-46e2-bf91-0c40007a31bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:34.189802 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c577f9-138f-46e2-bf91-0c40007a31bb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:34.190086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c577f9-138f-46e2-bf91-0c40007a31bb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:34.190362 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c577f9-138f-46e2-bf91-0c40007a31bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:34.191187 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c577f9-138f-46e2-bf91-0c40007a31bb None admin] RBAC: Adding query filter params (role.id=039f54fd5a574af19dad80e091f25108, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=94d824bffc2c4af299c537056bc21734) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:34.191500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c577f9-138f-46e2-bf91-0c40007a31bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:34.192956 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c577f9-138f-46e2-bf91-0c40007a31bb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:34.208058 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 498/982] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:56:34 2022] GET /identity/v3/role_assignments?user.id=94d824bffc2c4af299c537056bc21734&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=039f54fd5a574af19dad80e091f25108 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:34.501709 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82af4f6-f6de-43c9-a299-41995fc8109f None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:34.502294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82af4f6-f6de-43c9-a299-41995fc8109f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:34.502294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82af4f6-f6de-43c9-a299-41995fc8109f None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:34.503706 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 485/983] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:34.509303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd41d4d-e03c-430d-a951-f8aaec574474 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:34.509712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd41d4d-e03c-430d-a951-f8aaec574474 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:34.509890 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd41d4d-e03c-430d-a951-f8aaec574474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:34.527716 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3cd41d4d-e03c-430d-a951-f8aaec574474 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:34.528954 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3cd41d4d-e03c-430d-a951-f8aaec574474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:34.598496 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 499/984] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:34.621431 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82af4f6-f6de-43c9-a299-41995fc8109f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:34.704800 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:34.726697 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82af4f6-f6de-43c9-a299-41995fc8109f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:34.760378 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711974d9-0bde-4e9f-931f-312622dd4a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:34.761108 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711974d9-0bde-4e9f-931f-312622dd4a7c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:34.761397 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711974d9-0bde-4e9f-931f-312622dd4a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:34.761674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711974d9-0bde-4e9f-931f-312622dd4a7c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:34.762322 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711974d9-0bde-4e9f-931f-312622dd4a7c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:34.763691 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711974d9-0bde-4e9f-931f-312622dd4a7c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:34.775744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-711974d9-0bde-4e9f-931f-312622dd4a7c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 13:56:34.775744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:34.775744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:56:34.775744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:56:34.775744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:56:34.775744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:56:34.775744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:56:34.775744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:34.775744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 13:56:34.775744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:34.775744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:34.775744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:34.775744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:34.775744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:34.775744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:34.775744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:34.775744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:34.776910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:34.776910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:34.776910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:34.776910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:34.776910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:34.776910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:34.776910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:56:34.776910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:56:34.776910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:56:34.776910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:56:34.776910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:34.776910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:34.776910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:34.776910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:34.776910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:34.776910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:34.777795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:34.777795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:34.777795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:34.777795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:34.777795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:34.777795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:34.777795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:34.777795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:34.777795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:34.777795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:34.777795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:56:34.777795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:34.777795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 13:56:34.777795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:34.779284 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 486/985] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:56:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:34.782916 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd41d4d-e03c-430d-a951-f8aaec574474 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:34.842644 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b417468c-8e49-4376-968e-afdbdbb450ad None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:34.842941 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b417468c-8e49-4376-968e-afdbdbb450ad None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:34.843232 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b417468c-8e49-4376-968e-afdbdbb450ad None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:34.844769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 487/986] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:56:34.849004 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:34.849686 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01005a-d999-46e7-9537-cf94ddaa149c None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:34.850013 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01005a-d999-46e7-9537-cf94ddaa149c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:34.850297 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01005a-d999-46e7-9537-cf94ddaa149c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:34.867764 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-4f01005a-d999-46e7-9537-cf94ddaa149c None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:34.868992 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4f01005a-d999-46e7-9537-cf94ddaa149c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:34.895309 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd41d4d-e03c-430d-a951-f8aaec574474 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:34.934731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04bca23-e58b-411d-b3dc-6c1aceffdbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:34.935403 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bca23-e58b-411d-b3dc-6c1aceffdbb2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:34.935693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bca23-e58b-411d-b3dc-6c1aceffdbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:34.935988 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04bca23-e58b-411d-b3dc-6c1aceffdbb2 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:34.939616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04bca23-e58b-411d-b3dc-6c1aceffdbb2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:34.939616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04bca23-e58b-411d-b3dc-6c1aceffdbb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:34.939616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04bca23-e58b-411d-b3dc-6c1aceffdbb2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:34.946724 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 488/987] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 13:56:34.953162 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 500/988] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Jul 18 13:56:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:34.955010 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f01005a-d999-46e7-9537-cf94ddaa149c None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:34.969144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04bca23-e58b-411d-b3dc-6c1aceffdbb2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:35.033654 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:35.058688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f01005a-d999-46e7-9537-cf94ddaa149c None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:35.085998 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04bca23-e58b-411d-b3dc-6c1aceffdbb2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:35.093635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ef6e0-87cb-4ddd-a31f-e99cd7bdf508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:35.094306 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ef6e0-87cb-4ddd-a31f-e99cd7bdf508 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:35.094599 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ef6e0-87cb-4ddd-a31f-e99cd7bdf508 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:35.094881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ef6e0-87cb-4ddd-a31f-e99cd7bdf508 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:35.095593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ef6e0-87cb-4ddd-a31f-e99cd7bdf508 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:35.095903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ef6e0-87cb-4ddd-a31f-e99cd7bdf508 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:35.097304 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ef6e0-87cb-4ddd-a31f-e99cd7bdf508 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:35.102187 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 489/989] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Jul 18 13:56:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:35.107991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ef6e0-87cb-4ddd-a31f-e99cd7bdf508 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:35.115179 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:35.129302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da7681-384e-4383-bd05-486c83378c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:35.130278 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da7681-384e-4383-bd05-486c83378c45 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:35.130827 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da7681-384e-4383-bd05-486c83378c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:35.130827 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da7681-384e-4383-bd05-486c83378c45 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:35.132058 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da7681-384e-4383-bd05-486c83378c45 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:35.133438 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da7681-384e-4383-bd05-486c83378c45 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:35.146994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-58da7681-384e-4383-bd05-486c83378c45 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 13:56:35.146994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:35.146994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:56:35.146994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:56:35.146994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:56:35.146994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:56:35.146994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:56:35.146994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:35.146994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 13:56:35.146994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:35.146994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:35.146994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:35.146994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:35.146994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:35.146994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:35.146994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:35.146994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:35.148266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:35.148266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:35.148266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:35.148266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:35.148266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:35.148266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:35.148266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:56:35.148266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:56:35.148266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:56:35.148266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:56:35.148266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:35.148266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:35.148266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:35.148266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:35.148266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:35.148266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:35.149332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:35.149332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:35.149332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:35.149332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:35.149332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:35.149332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:35.149332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:35.149332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:35.149332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:35.149332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:35.149332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:56:35.149332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:35.149332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 13:56:35.149332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:35.151411 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 501/990] 10.222.0.94 () {56 vars in 1126 bytes} [Mon Jul 18 13:56:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:35.157238 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da7681-384e-4383-bd05-486c83378c45 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:35.215633 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ef6e0-87cb-4ddd-a31f-e99cd7bdf508 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:35.255732 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4637d7-e830-4e01-a42f-64bb8f7b5172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:35.257084 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4637d7-e830-4e01-a42f-64bb8f7b5172 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:35.257375 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4637d7-e830-4e01-a42f-64bb8f7b5172 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:35.257809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4637d7-e830-4e01-a42f-64bb8f7b5172 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:35.258546 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4637d7-e830-4e01-a42f-64bb8f7b5172 None admin] RBAC: Adding query filter params (interface=public, service_id=8dab408b12684b60851c890b29df67d3, region_id=RegionOne) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:35.258861 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4637d7-e830-4e01-a42f-64bb8f7b5172 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:35.260385 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4637d7-e830-4e01-a42f-64bb8f7b5172 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:35.263389 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da7681-384e-4383-bd05-486c83378c45 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:35.266005 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 490/991] 10.222.0.94 () {56 vars in 1251 bytes} [Mon Jul 18 13:56:35 2022] GET /identity/v3/endpoints?service_id=8dab408b12684b60851c890b29df67d3&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:35.271364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4637d7-e830-4e01-a42f-64bb8f7b5172 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:35.276182 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:35.302330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261febb6-9f9d-4ca9-812c-90b3a9897a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:35.303567 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261febb6-9f9d-4ca9-812c-90b3a9897a43 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:35.303861 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261febb6-9f9d-4ca9-812c-90b3a9897a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:35.304170 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261febb6-9f9d-4ca9-812c-90b3a9897a43 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:35.305216 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261febb6-9f9d-4ca9-812c-90b3a9897a43 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:35.305813 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261febb6-9f9d-4ca9-812c-90b3a9897a43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:35.307359 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261febb6-9f9d-4ca9-812c-90b3a9897a43 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:35.318479 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 502/992] 10.222.0.94 () {56 vars in 1122 bytes} [Mon Jul 18 13:56:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:35.320375 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261febb6-9f9d-4ca9-812c-90b3a9897a43 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:35.376397 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:35.387503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4637d7-e830-4e01-a42f-64bb8f7b5172 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:35.427101 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fde52-1086-48bf-b839-74473f3cdc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:35.427773 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fde52-1086-48bf-b839-74473f3cdc8d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:35.428640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fde52-1086-48bf-b839-74473f3cdc8d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:35.428919 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87fde52-1086-48bf-b839-74473f3cdc8d None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:35.429319 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261febb6-9f9d-4ca9-812c-90b3a9897a43 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:35.429581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87fde52-1086-48bf-b839-74473f3cdc8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:35.430949 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87fde52-1086-48bf-b839-74473f3cdc8d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:35.437491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 491/993] 10.222.0.94 () {56 vars in 1087 bytes} [Mon Jul 18 13:56:35 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 13:56:35.467755 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06325537-833b-4492-93a2-e223de370f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:35.468483 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06325537-833b-4492-93a2-e223de370f00 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:35.468784 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06325537-833b-4492-93a2-e223de370f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:35.469054 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06325537-833b-4492-93a2-e223de370f00 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:35.469786 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06325537-833b-4492-93a2-e223de370f00 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:35.470103 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06325537-833b-4492-93a2-e223de370f00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:35.471518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06325537-833b-4492-93a2-e223de370f00 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:35.477844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 503/994] 10.222.0.94 () {56 vars in 1122 bytes} [Mon Jul 18 13:56:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:35.483581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fde52-1086-48bf-b839-74473f3cdc8d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:35.556509 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:35.594586 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87fde52-1086-48bf-b839-74473f3cdc8d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:35.632513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff0a66-2b67-4e04-bb39-f5877eae45ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:35.633512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff0a66-2b67-4e04-bb39-f5877eae45ba None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:35.633980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff0a66-2b67-4e04-bb39-f5877eae45ba None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:35.634439 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff0a66-2b67-4e04-bb39-f5877eae45ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:35.635422 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff0a66-2b67-4e04-bb39-f5877eae45ba None admin] RBAC: Adding query filter params (interface=admin, service_id=d65d96de56554680a009f71a30e44c6a, region_id=RegionOne) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:35.635928 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff0a66-2b67-4e04-bb39-f5877eae45ba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:35.637615 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff0a66-2b67-4e04-bb39-f5877eae45ba None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:35.643531 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 492/995] 10.222.0.94 () {56 vars in 1249 bytes} [Mon Jul 18 13:56:35 2022] GET /identity/v3/endpoints?service_id=d65d96de56554680a009f71a30e44c6a&interface=admin®ion_id=RegionOne => generated 193 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:35.648822 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06325537-833b-4492-93a2-e223de370f00 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:35.752021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06325537-833b-4492-93a2-e223de370f00 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:35.765834 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:35.786801 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eae588-a879-42de-bcd0-1bb436a7d052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:35.788045 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eae588-a879-42de-bcd0-1bb436a7d052 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:35.788339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eae588-a879-42de-bcd0-1bb436a7d052 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:35.788620 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eae588-a879-42de-bcd0-1bb436a7d052 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:35.789309 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eae588-a879-42de-bcd0-1bb436a7d052 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:35.790691 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eae588-a879-42de-bcd0-1bb436a7d052 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:35.803966 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 504/996] 10.222.0.94 () {56 vars in 1087 bytes} [Mon Jul 18 13:56:35 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 13:56:36.232713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eb039-564c-44bc-ab4c-6868dc1999f3 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:36.233083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eb039-564c-44bc-ab4c-6868dc1999f3 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:36.233199 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eb039-564c-44bc-ab4c-6868dc1999f3 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:36.234605 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 493/997] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:36.239777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0ea6f-cc6a-4163-8dc6-b8842df2e1ee None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:36.240184 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0ea6f-cc6a-4163-8dc6-b8842df2e1ee None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:36.240533 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0ea6f-cc6a-4163-8dc6-b8842df2e1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:36.258510 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d3f0ea6f-cc6a-4163-8dc6-b8842df2e1ee None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:36.259724 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d3f0ea6f-cc6a-4163-8dc6-b8842df2e1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:36.328958 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 505/998] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:36.351230 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501eb039-564c-44bc-ab4c-6868dc1999f3 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:36.462370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501eb039-564c-44bc-ab4c-6868dc1999f3 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:36.492701 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:36.499209 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06d8eb-a57a-4a6a-8432-71413ff6139f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:36.500029 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06d8eb-a57a-4a6a-8432-71413ff6139f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:36.500318 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06d8eb-a57a-4a6a-8432-71413ff6139f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:36.500597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06d8eb-a57a-4a6a-8432-71413ff6139f None admin] PATH_INFO: `/v3/services/image` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:36.501253 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa06d8eb-a57a-4a6a-8432-71413ff6139f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:36.502618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa06d8eb-a57a-4a6a-8432-71413ff6139f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:36.514416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-aa06d8eb-a57a-4a6a-8432-71413ff6139f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 13:56:36.514416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:36.514416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:56:36.514416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:56:36.514416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:56:36.514416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:56:36.514416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:56:36.514416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:36.514416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 13:56:36.514416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:36.514416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:36.514416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:36.514416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:36.514416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:36.514416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:36.514416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:36.514416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:36.515570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:36.515570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:36.515570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:36.515570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:36.515570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:36.515570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:36.515570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:56:36.515570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:56:36.515570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:56:36.515570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:56:36.515570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:36.515570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:36.515570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:36.515570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:36.515570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:36.515570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:36.516538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:36.516538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:36.516538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:36.516538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:36.516538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:36.516538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:36.516538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:36.516538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:36.516538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:36.516538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:36.516538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:56:36.516538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:36.516538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 13:56:36.516538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:36.516538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 494/999] 10.222.0.94 () {56 vars in 1105 bytes} [Mon Jul 18 13:56:36 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:36.523582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0ea6f-cc6a-4163-8dc6-b8842df2e1ee None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:36.633109 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0ea6f-cc6a-4163-8dc6-b8842df2e1ee None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:36.642857 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:36.670931 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5569298-d3e1-4919-b878-88532908e876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:36.671676 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5569298-d3e1-4919-b878-88532908e876 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:36.671965 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5569298-d3e1-4919-b878-88532908e876 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:36.672257 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5569298-d3e1-4919-b878-88532908e876 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:36.673126 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5569298-d3e1-4919-b878-88532908e876 None admin] RBAC: Adding query filter params (name=image) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:36.673476 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5569298-d3e1-4919-b878-88532908e876 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:36.674897 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5569298-d3e1-4919-b878-88532908e876 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:36.684635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 506/1000] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:36.686036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06d8eb-a57a-4a6a-8432-71413ff6139f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:36.791160 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06d8eb-a57a-4a6a-8432-71413ff6139f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:36.825535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67308172-daf4-4703-982a-9fd2a7981fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:36.826256 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67308172-daf4-4703-982a-9fd2a7981fcc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:36.826589 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67308172-daf4-4703-982a-9fd2a7981fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:36.826909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67308172-daf4-4703-982a-9fd2a7981fcc None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:36.827660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67308172-daf4-4703-982a-9fd2a7981fcc None admin] RBAC: Adding query filter params (type=image) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:36.828001 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67308172-daf4-4703-982a-9fd2a7981fcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:36.829785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67308172-daf4-4703-982a-9fd2a7981fcc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:36.840178 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 495/1001] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:36 2022] GET /identity/v3/services?type=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:37.506764 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9c208-e90a-4614-8d83-8924d31cc2b2 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:37.507124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9c208-e90a-4614-8d83-8924d31cc2b2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:37.507338 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9c208-e90a-4614-8d83-8924d31cc2b2 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:37.508804 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 507/1002] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:37.513894 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73282d-0d80-4ee7-ada6-6a0a51c1ce98 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:37.514193 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73282d-0d80-4ee7-ada6-6a0a51c1ce98 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:37.514480 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73282d-0d80-4ee7-ada6-6a0a51c1ce98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:37.531460 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-9c73282d-0d80-4ee7-ada6-6a0a51c1ce98 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:37.532666 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9c73282d-0d80-4ee7-ada6-6a0a51c1ce98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:37.589132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:37.604441 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 496/1003] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:37.626754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9c208-e90a-4614-8d83-8924d31cc2b2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:37.728997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9c208-e90a-4614-8d83-8924d31cc2b2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:37.762394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db84cde-e9fc-44c4-8b5e-e4f6aedf57b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:37.763112 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db84cde-e9fc-44c4-8b5e-e4f6aedf57b3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:37.763395 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db84cde-e9fc-44c4-8b5e-e4f6aedf57b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:37.763675 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db84cde-e9fc-44c4-8b5e-e4f6aedf57b3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:37.764366 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db84cde-e9fc-44c4-8b5e-e4f6aedf57b3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:37.765734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db84cde-e9fc-44c4-8b5e-e4f6aedf57b3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:37.777848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-9db84cde-e9fc-44c4-8b5e-e4f6aedf57b3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 13:56:37.777848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:37.777848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:56:37.777848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:56:37.777848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:56:37.777848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:56:37.777848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:56:37.777848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:37.777848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 13:56:37.777848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:37.777848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:37.777848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:37.777848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:37.777848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:37.777848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:37.777848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:37.777848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:37.778836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:37.778836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:37.778836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:37.778836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:37.778836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:37.778836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:37.778836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:56:37.778836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:56:37.778836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:56:37.778836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:56:37.778836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:37.778836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:37.778836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:37.778836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:37.778836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:37.778836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:37.779700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:37.779700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:37.779700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:37.779700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:37.779700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:37.779700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:37.779700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:37.779700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:37.779700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:37.779700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:37.779700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:56:37.779700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:37.779700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 13:56:37.779700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:37.781126 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 508/1004] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:56:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:37.785063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73282d-0d80-4ee7-ada6-6a0a51c1ce98 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:37.807087 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a930d31-66da-4dcb-b439-dc29cf59293e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:37.807377 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a930d31-66da-4dcb-b439-dc29cf59293e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:37.807663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a930d31-66da-4dcb-b439-dc29cf59293e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:37.809272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 509/1005] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:56:37.814351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10903a6e-70a2-4cca-8a7f-c0be5d3f6180 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:37.814639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10903a6e-70a2-4cca-8a7f-c0be5d3f6180 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:37.814917 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10903a6e-70a2-4cca-8a7f-c0be5d3f6180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:37.832608 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-10903a6e-70a2-4cca-8a7f-c0be5d3f6180 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:37.833797 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-10903a6e-70a2-4cca-8a7f-c0be5d3f6180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:37.863103 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:37.893753 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73282d-0d80-4ee7-ada6-6a0a51c1ce98 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:37.909949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 510/1006] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 13:56:37.932346 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10903a6e-70a2-4cca-8a7f-c0be5d3f6180 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:37.932989 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5024d2a-39c5-4c0c-bf14-d0d26c8e3d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:37.933687 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5024d2a-39c5-4c0c-bf14-d0d26c8e3d17 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:37.933973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5024d2a-39c5-4c0c-bf14-d0d26c8e3d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:37.934251 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5024d2a-39c5-4c0c-bf14-d0d26c8e3d17 None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:37.934981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5024d2a-39c5-4c0c-bf14-d0d26c8e3d17 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:37.935287 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5024d2a-39c5-4c0c-bf14-d0d26c8e3d17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:37.939502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5024d2a-39c5-4c0c-bf14-d0d26c8e3d17 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:37.943188 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 497/1007] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Jul 18 13:56:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:37.949409 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5024d2a-39c5-4c0c-bf14-d0d26c8e3d17 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:38.009556 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:38.053604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10903a6e-70a2-4cca-8a7f-c0be5d3f6180 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:38.060067 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5024d2a-39c5-4c0c-bf14-d0d26c8e3d17 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:38.089300 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665966c-bde0-4374-b39a-b04f91f6127c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:38.090114 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665966c-bde0-4374-b39a-b04f91f6127c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:38.090409 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665966c-bde0-4374-b39a-b04f91f6127c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:38.090689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665966c-bde0-4374-b39a-b04f91f6127c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:38.091344 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665966c-bde0-4374-b39a-b04f91f6127c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:38.092750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8665966c-bde0-4374-b39a-b04f91f6127c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:38.096024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c30e96-f970-4664-b499-3b986bab740c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:38.096689 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c30e96-f970-4664-b499-3b986bab740c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:38.096977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c30e96-f970-4664-b499-3b986bab740c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:38.097257 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c30e96-f970-4664-b499-3b986bab740c None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:38.097959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c30e96-f970-4664-b499-3b986bab740c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:38.098265 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c30e96-f970-4664-b499-3b986bab740c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:38.099640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c30e96-f970-4664-b499-3b986bab740c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:38.104703 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 498/1008] 10.222.0.94 () {56 vars in 1112 bytes} [Mon Jul 18 13:56:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:38.105255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-8665966c-bde0-4374-b39a-b04f91f6127c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 13:56:38.105255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:38.105255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:56:38.105255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:56:38.105255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:56:38.105255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:56:38.105255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:56:38.105255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:38.105255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 13:56:38.105255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:38.105255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:38.105255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:38.105255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:38.105255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:38.105255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:38.105255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:38.105255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:38.111518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:38.111518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:38.111518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:38.111518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:38.111518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:38.111518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:38.111518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:56:38.111518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:56:38.111518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:56:38.111518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:56:38.111518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:38.111518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:38.111518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:38.111518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:38.111518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:38.111518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:38.112516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:38.112516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:38.112516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:38.112516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:38.112516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:38.112516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:38.112516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:38.112516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:38.112516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:38.112516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:38.112516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:56:38.112516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:38.112516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 13:56:38.112516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:38.112516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 511/1009] 10.222.0.94 () {56 vars in 1126 bytes} [Mon Jul 18 13:56:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:38.113318 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c30e96-f970-4664-b499-3b986bab740c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:38.113399 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665966c-bde0-4374-b39a-b04f91f6127c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:38.145709 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:38.219502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c30e96-f970-4664-b499-3b986bab740c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:38.220989 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665966c-bde0-4374-b39a-b04f91f6127c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:38.236225 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:38.256545 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0413855-b146-456d-961d-2cedb5402ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:38.257205 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0413855-b146-456d-961d-2cedb5402ca4 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:38.257487 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0413855-b146-456d-961d-2cedb5402ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:38.257759 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0413855-b146-456d-961d-2cedb5402ca4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:38.258075 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70352e53-b821-4702-bdfa-5439fc8a1ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:38.258500 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0413855-b146-456d-961d-2cedb5402ca4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:38.258744 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70352e53-b821-4702-bdfa-5439fc8a1ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:38.259034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70352e53-b821-4702-bdfa-5439fc8a1ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:38.259312 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70352e53-b821-4702-bdfa-5439fc8a1ce5 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:38.259854 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0413855-b146-456d-961d-2cedb5402ca4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:38.260057 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70352e53-b821-4702-bdfa-5439fc8a1ce5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:38.260357 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70352e53-b821-4702-bdfa-5439fc8a1ce5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:38.261716 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70352e53-b821-4702-bdfa-5439fc8a1ce5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:38.266931 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 512/1010] 10.222.0.94 () {56 vars in 1122 bytes} [Mon Jul 18 13:56:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:38.272648 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70352e53-b821-4702-bdfa-5439fc8a1ce5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:38.287971 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 499/1011] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Jul 18 13:56:38 2022] POST /identity/v3/endpoints => generated 334 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 13:56:38.376700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:38.385057 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70352e53-b821-4702-bdfa-5439fc8a1ce5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:38.428791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a21a6e-d0a6-47df-a602-f5c61cbe4734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:38.429001 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a21a6e-d0a6-47df-a602-f5c61cbe4734 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:38.429201 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a21a6e-d0a6-47df-a602-f5c61cbe4734 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:38.429419 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a21a6e-d0a6-47df-a602-f5c61cbe4734 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:38.430151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a21a6e-d0a6-47df-a602-f5c61cbe4734 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:38.430428 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a21a6e-d0a6-47df-a602-f5c61cbe4734 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:38.431779 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a21a6e-d0a6-47df-a602-f5c61cbe4734 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:38.437286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 513/1012] 10.222.0.94 () {56 vars in 1122 bytes} [Mon Jul 18 13:56:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:38.442563 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0413855-b146-456d-961d-2cedb5402ca4 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:38.545288 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0413855-b146-456d-961d-2cedb5402ca4 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:38.580298 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a933f02-843f-4309-89fa-47ece4a6dcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:38.580967 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a933f02-843f-4309-89fa-47ece4a6dcaa None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:38.581264 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a933f02-843f-4309-89fa-47ece4a6dcaa None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:38.581549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a933f02-843f-4309-89fa-47ece4a6dcaa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:38.582293 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a933f02-843f-4309-89fa-47ece4a6dcaa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:38.583666 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a933f02-843f-4309-89fa-47ece4a6dcaa None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:38.600052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 500/1013] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Jul 18 13:56:38 2022] POST /identity/v3/endpoints => generated 325 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:56:38.846888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53812a-68db-4675-a43a-62d78a2a6455 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:38.847413 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53812a-68db-4675-a43a-62d78a2a6455 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:38.847413 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53812a-68db-4675-a43a-62d78a2a6455 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:38.848902 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 514/1014] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:38.854490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb36a9c-2cf1-4f62-8799-54a25d625226 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:38.854921 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb36a9c-2cf1-4f62-8799-54a25d625226 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:38.855140 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb36a9c-2cf1-4f62-8799-54a25d625226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:38.872211 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-acb36a9c-2cf1-4f62-8799-54a25d625226 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:38.873396 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-acb36a9c-2cf1-4f62-8799-54a25d625226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:38.900889 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:38.959984 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 501/1015] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:38 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:38.979976 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53812a-68db-4675-a43a-62d78a2a6455 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:39.083563 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53812a-68db-4675-a43a-62d78a2a6455 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:39.117510 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4db5e7-c61f-45ef-8c16-0d908f056964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:39.118100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4db5e7-c61f-45ef-8c16-0d908f056964 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:39.118386 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4db5e7-c61f-45ef-8c16-0d908f056964 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:39.118662 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4db5e7-c61f-45ef-8c16-0d908f056964 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:39.119388 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4db5e7-c61f-45ef-8c16-0d908f056964 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:39.120792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4db5e7-c61f-45ef-8c16-0d908f056964 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:39.143975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 515/1016] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Jul 18 13:56:38 2022] POST /identity/v3/services => generated 242 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:56:40.398648 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26002c8e-2be3-4aeb-8ee0-46aec476f18a None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:40.399154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26002c8e-2be3-4aeb-8ee0-46aec476f18a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:40.399232 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26002c8e-2be3-4aeb-8ee0-46aec476f18a None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:40.401600 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 502/1017] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:40.407605 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8d540-21a1-4911-beae-635ddedb4741 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:40.408057 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8d540-21a1-4911-beae-635ddedb4741 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:40.408231 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8d540-21a1-4911-beae-635ddedb4741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:40.424630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-32d8d540-21a1-4911-beae-635ddedb4741 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:40.425810 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-32d8d540-21a1-4911-beae-635ddedb4741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:40.436018 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:40.503855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 516/1018] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:40.527356 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26002c8e-2be3-4aeb-8ee0-46aec476f18a None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:40.629244 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26002c8e-2be3-4aeb-8ee0-46aec476f18a None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:40.663505 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f588f6d-5977-4e32-a10c-b1cd82da7f91 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:40.663718 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f588f6d-5977-4e32-a10c-b1cd82da7f91 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:40.664047 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f588f6d-5977-4e32-a10c-b1cd82da7f91 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:40.665562 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 517/1019] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:56:40.669277 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae73c55-89c3-448c-ae1b-61f4e1c3f344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:40.670046 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae73c55-89c3-448c-ae1b-61f4e1c3f344 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:40.670331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae73c55-89c3-448c-ae1b-61f4e1c3f344 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:40.670438 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fce278-1185-406b-ac5f-10e7bc2a874c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:40.670610 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae73c55-89c3-448c-ae1b-61f4e1c3f344 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:40.670707 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fce278-1185-406b-ac5f-10e7bc2a874c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:40.670989 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fce278-1185-406b-ac5f-10e7bc2a874c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:40.682066 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae73c55-89c3-448c-ae1b-61f4e1c3f344 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:56:40.688285 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-66fce278-1185-406b-ac5f-10e7bc2a874c None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:40.689485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-66fce278-1185-406b-ac5f-10e7bc2a874c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:40.690438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-4ae73c55-89c3-448c-ae1b-61f4e1c3f344 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 13:56:40.690438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:40.690438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:40.690438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:40.690438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:40.690438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:40.690438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:40.690438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:40.690438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:40.690438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:40.690438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:40.690438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:40.690438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:56:40.690438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:56:40.690438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:56:40.690438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:56:40.691373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:56:40.691373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:56:40.691373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:56:40.691373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:56:40.691373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:40.691373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:40.691373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:40.691373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:40.691373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:40.691373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:40.691373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:40.691373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:40.691373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:40.691373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:40.691373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:40.691373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:40.692987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:40.692987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:40.692987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:40.692987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:40.692987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:56:40.692987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:56:40.692987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:56:40.692987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:56:40.692987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:56:40.692987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:56:40.692987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 13:56:40.692987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:40.692987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 503/1020] 10.222.0.94 () {56 vars in 1120 bytes} [Mon Jul 18 13:56:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:40.698026 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae73c55-89c3-448c-ae1b-61f4e1c3f344 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:40.717847 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:40.764128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 518/1021] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 13:56:40.788176 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fce278-1185-406b-ac5f-10e7bc2a874c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:40.809536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae73c55-89c3-448c-ae1b-61f4e1c3f344 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:40.846013 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567873bd-cf6f-426b-9d99-bfb9af5ac919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:40.846692 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567873bd-cf6f-426b-9d99-bfb9af5ac919 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:40.846981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567873bd-cf6f-426b-9d99-bfb9af5ac919 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:40.847264 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567873bd-cf6f-426b-9d99-bfb9af5ac919 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:40.848033 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567873bd-cf6f-426b-9d99-bfb9af5ac919 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:40.848350 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567873bd-cf6f-426b-9d99-bfb9af5ac919 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:40.849760 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567873bd-cf6f-426b-9d99-bfb9af5ac919 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:40.858040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 504/1022] 10.222.0.94 () {56 vars in 1115 bytes} [Mon Jul 18 13:56:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:40.863689 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567873bd-cf6f-426b-9d99-bfb9af5ac919 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:40.867565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:40.902288 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fce278-1185-406b-ac5f-10e7bc2a874c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:40.939614 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c374d0-4de7-4b88-83db-15543ed5069c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:40.940392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c374d0-4de7-4b88-83db-15543ed5069c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:40.940684 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c374d0-4de7-4b88-83db-15543ed5069c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:40.940971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c374d0-4de7-4b88-83db-15543ed5069c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:40.942194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c374d0-4de7-4b88-83db-15543ed5069c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:40.944229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c374d0-4de7-4b88-83db-15543ed5069c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:40.946165 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 519/1023] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:40.950957 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:40.951814 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c374d0-4de7-4b88-83db-15543ed5069c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:40.972780 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567873bd-cf6f-426b-9d99-bfb9af5ac919 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:41.007800 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9e4bf-dc91-410d-a8f4-d2fb1c089b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:41.008622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9e4bf-dc91-410d-a8f4-d2fb1c089b81 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:41.008908 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9e4bf-dc91-410d-a8f4-d2fb1c089b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:41.009243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9e4bf-dc91-410d-a8f4-d2fb1c089b81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:41.020156 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9e4bf-dc91-410d-a8f4-d2fb1c089b81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:41.022158 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9e4bf-dc91-410d-a8f4-d2fb1c089b81 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:41.041050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-2cc9e4bf-dc91-410d-a8f4-d2fb1c089b81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:41.041050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:41.041050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:41.041050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:41.041050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:41.041050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:41.041050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:41.041050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:41.041050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:41.041050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:41.041050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:41.041050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:41.041050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:41.041050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:41.041050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:41.041979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:41.041979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:41.041979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:41.041979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:41.041979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:41.041979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:41.041979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:41.041979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:41.041979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:41.041979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:41.041979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:41.041979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:41.041979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:41.041979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:41.041979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:41.041979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:41.041979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:41.047577 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:41.047577 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:41.047577 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:41.047577 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:41.047577 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 505/1024] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:41.047861 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9e4bf-dc91-410d-a8f4-d2fb1c089b81 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:41.060864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c374d0-4de7-4b88-83db-15543ed5069c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:41.096777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab3c56-74d4-4be4-89bd-45880e928d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:41.097441 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab3c56-74d4-4be4-89bd-45880e928d91 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:41.097738 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab3c56-74d4-4be4-89bd-45880e928d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:41.098023 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab3c56-74d4-4be4-89bd-45880e928d91 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:41.098732 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab3c56-74d4-4be4-89bd-45880e928d91 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:41.100203 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab3c56-74d4-4be4-89bd-45880e928d91 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:41.108411 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:41.136181 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 520/1025] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Jul 18 13:56:40 2022] POST /identity/v3/projects => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 13:56:41.159291 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9e4bf-dc91-410d-a8f4-d2fb1c089b81 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:41.191856 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:41.198215 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508be04d-b10f-4617-ae8c-0ad11cb85e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:41.198427 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27168c28-3b3b-49b5-b9e0-323b2ca3017e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:41.198687 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27168c28-3b3b-49b5-b9e0-323b2ca3017e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:41.198904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508be04d-b10f-4617-ae8c-0ad11cb85e87 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:41.199013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27168c28-3b3b-49b5-b9e0-323b2ca3017e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:41.199183 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508be04d-b10f-4617-ae8c-0ad11cb85e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:41.199461 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508be04d-b10f-4617-ae8c-0ad11cb85e87 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:41.200233 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508be04d-b10f-4617-ae8c-0ad11cb85e87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:41.203973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508be04d-b10f-4617-ae8c-0ad11cb85e87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:41.204177 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 521/1026] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:56:41.204290 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508be04d-b10f-4617-ae8c-0ad11cb85e87 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:41.206615 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1debd40-fb5c-423a-8198-9e8cf33c61fc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:41.206955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1debd40-fb5c-423a-8198-9e8cf33c61fc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:41.207240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1debd40-fb5c-423a-8198-9e8cf33c61fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:41.217869 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 506/1027] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:41.224061 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508be04d-b10f-4617-ae8c-0ad11cb85e87 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:41.228630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b1debd40-fb5c-423a-8198-9e8cf33c61fc None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:41.229864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b1debd40-fb5c-423a-8198-9e8cf33c61fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:41.305392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 522/1028] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 13:56:41.329629 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1debd40-fb5c-423a-8198-9e8cf33c61fc None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:41.345629 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508be04d-b10f-4617-ae8c-0ad11cb85e87 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:41.381565 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4cb09-2936-4b22-a97b-568ad12f663d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:41.382361 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4cb09-2936-4b22-a97b-568ad12f663d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:41.382649 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4cb09-2936-4b22-a97b-568ad12f663d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:41.382928 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4cb09-2936-4b22-a97b-568ad12f663d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:41.390327 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4cb09-2936-4b22-a97b-568ad12f663d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:41.391782 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4cb09-2936-4b22-a97b-568ad12f663d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:41.410503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-eab4cb09-2936-4b22-a97b-568ad12f663d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 13:56:41.410503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:41.410503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:41.410503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:41.410503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:41.410503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:41.410503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:41.410503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:41.410503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:41.410503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:41.410503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:41.410503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:41.410503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:56:41.410503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:56:41.410503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:56:41.410503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:56:41.412180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:41.412180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:41.412180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:56:41.412180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:41.412180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:56:41.412180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:41.412180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:41.412180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:41.412180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:41.412180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:41.412180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:41.412180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:41.412180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:41.412180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:41.412180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:41.412180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:41.412180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:41.412180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:41.413337 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:41.413337 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:41.413337 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:56:41.413337 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:56:41.413337 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:56:41.413337 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:56:41.413337 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:56:41.413337 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:56:41.413337 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 13:56:41.413337 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:41.415284 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 507/1029] 10.222.0.94 () {56 vars in 1093 bytes} [Mon Jul 18 13:56:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:41.420834 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4cb09-2936-4b22-a97b-568ad12f663d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:41.440119 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1debd40-fb5c-423a-8198-9e8cf33c61fc None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:41.461265 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:41.477358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d388d05-bbf5-4359-888a-a5485a41ee0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:41.478259 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d388d05-bbf5-4359-888a-a5485a41ee0e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:41.478617 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d388d05-bbf5-4359-888a-a5485a41ee0e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:41.478964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d388d05-bbf5-4359-888a-a5485a41ee0e None admin] PATH_INFO: `/v3/services/image` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:41.479700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d388d05-bbf5-4359-888a-a5485a41ee0e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:41.479834 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:41.481449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d388d05-bbf5-4359-888a-a5485a41ee0e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:41.494392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-8d388d05-bbf5-4359-888a-a5485a41ee0e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 13:56:41.494392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:41.494392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:56:41.494392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:56:41.494392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:56:41.494392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:56:41.494392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:56:41.494392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:41.494392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 13:56:41.494392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:41.494392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:41.494392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:41.494392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:41.494392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:41.494392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:41.494392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:41.494392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:41.495405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:41.495405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:41.495405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:41.495405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:41.495405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:41.495405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:41.495405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:56:41.495405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:56:41.495405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:56:41.495405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:56:41.495405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:41.495405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:41.495405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:41.495405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:41.495405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:41.495405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:41.496297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:41.496297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:41.496297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:41.496297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:41.496297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:41.496297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:41.496297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:41.496297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:41.496297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:41.496297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:41.496297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:56:41.496297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:41.496297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 13:56:41.496297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:41.496297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 523/1030] 10.222.0.94 () {56 vars in 1105 bytes} [Mon Jul 18 13:56:41 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:41.502126 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d388d05-bbf5-4359-888a-a5485a41ee0e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:41.534929 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4cb09-2936-4b22-a97b-568ad12f663d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:41.572052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46ba7b-4cb1-40b9-9949-9fff0ffcc2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:41.572727 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46ba7b-4cb1-40b9-9949-9fff0ffcc2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:41.573015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46ba7b-4cb1-40b9-9949-9fff0ffcc2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:41.573294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46ba7b-4cb1-40b9-9949-9fff0ffcc2e8 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:41.574134 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a46ba7b-4cb1-40b9-9949-9fff0ffcc2e8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:41.574693 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a46ba7b-4cb1-40b9-9949-9fff0ffcc2e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:41.575886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a46ba7b-4cb1-40b9-9949-9fff0ffcc2e8 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:41.595654 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 508/1031] 10.222.0.94 () {56 vars in 1133 bytes} [Mon Jul 18 13:56:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:41.601650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46ba7b-4cb1-40b9-9949-9fff0ffcc2e8 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:41.620184 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d388d05-bbf5-4359-888a-a5485a41ee0e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:41.657114 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41318f07-0772-4145-b2fe-f416c777ace5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:41.657790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41318f07-0772-4145-b2fe-f416c777ace5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:41.658081 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41318f07-0772-4145-b2fe-f416c777ace5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:41.658359 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41318f07-0772-4145-b2fe-f416c777ace5 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:41.659157 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41318f07-0772-4145-b2fe-f416c777ace5 None admin] RBAC: Adding query filter params (name=image) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:41.659525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41318f07-0772-4145-b2fe-f416c777ace5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:41.661114 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41318f07-0772-4145-b2fe-f416c777ace5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:41.666515 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 524/1032] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:41.672070 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41318f07-0772-4145-b2fe-f416c777ace5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:41.715413 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46ba7b-4cb1-40b9-9949-9fff0ffcc2e8 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:41.723872 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:41.727124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:41.755038 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d11be8-834e-4d19-a8b7-d12a17f072f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:41.756088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d11be8-834e-4d19-a8b7-d12a17f072f9 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:41.756376 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d11be8-834e-4d19-a8b7-d12a17f072f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:41.756652 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d11be8-834e-4d19-a8b7-d12a17f072f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:41.777946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41318f07-0772-4145-b2fe-f416c777ace5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:41.813601 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd7024-70cd-44df-9a54-a4b1e5fc9b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:41.814365 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd7024-70cd-44df-9a54-a4b1e5fc9b5b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:41.814661 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd7024-70cd-44df-9a54-a4b1e5fc9b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:41.814943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd7024-70cd-44df-9a54-a4b1e5fc9b5b None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:41.815650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd7024-70cd-44df-9a54-a4b1e5fc9b5b None admin] RBAC: Adding query filter params (type=image) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:41.815987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd7024-70cd-44df-9a54-a4b1e5fc9b5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:41.817363 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd7024-70cd-44df-9a54-a4b1e5fc9b5b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:41.835786 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 525/1033] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:41.837122 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd7024-70cd-44df-9a54-a4b1e5fc9b5b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:41.839010 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d11be8-834e-4d19-a8b7-d12a17f072f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:41.840770 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d11be8-834e-4d19-a8b7-d12a17f072f9 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:41.956057 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd7024-70cd-44df-9a54-a4b1e5fc9b5b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:41.973788 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 509/1034] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:56:41 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 13:56:41.974012 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:41.975517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d11be8-834e-4d19-a8b7-d12a17f072f9 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:41.987225 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:42.002567 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e3b1f-b094-4704-99ea-2c26f98b278b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:42.003381 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e3b1f-b094-4704-99ea-2c26f98b278b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:42.003670 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e3b1f-b094-4704-99ea-2c26f98b278b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:42.004299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e3b1f-b094-4704-99ea-2c26f98b278b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:42.005362 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e3b1f-b094-4704-99ea-2c26f98b278b None admin] RBAC: Adding query filter params (interface=public, service_id=0069f12a56754dd78abe4978ef08afd1, region_id=RegionOne) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:42.005707 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e3b1f-b094-4704-99ea-2c26f98b278b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:42.007536 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e3b1f-b094-4704-99ea-2c26f98b278b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:42.018311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 526/1035] 10.222.0.94 () {56 vars in 1251 bytes} [Mon Jul 18 13:56:41 2022] GET /identity/v3/endpoints?service_id=0069f12a56754dd78abe4978ef08afd1&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:42.019692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e3b1f-b094-4704-99ea-2c26f98b278b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:42.079906 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d11be8-834e-4d19-a8b7-d12a17f072f9 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:42.117401 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf4e15-55cf-4bb7-9fd7-8737a19d2e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:42.118201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf4e15-55cf-4bb7-9fd7-8737a19d2e33 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:42.118489 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf4e15-55cf-4bb7-9fd7-8737a19d2e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:42.118769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf4e15-55cf-4bb7-9fd7-8737a19d2e33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:42.129105 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf4e15-55cf-4bb7-9fd7-8737a19d2e33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:42.130622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf4e15-55cf-4bb7-9fd7-8737a19d2e33 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:42.134859 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e3b1f-b094-4704-99ea-2c26f98b278b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:42.148680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-cfaf4e15-55cf-4bb7-9fd7-8737a19d2e33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:42.148680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:42.148680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:42.148680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:42.148680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:42.148680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:42.148680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:42.148680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:42.148680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:42.148680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:42.148680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:42.148680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:42.148680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:42.148680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:42.148680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:42.149606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:42.149606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:42.149606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:42.149606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:42.149606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:42.149606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:42.149606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:42.149606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:42.149606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:42.149606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:42.149606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:42.149606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:42.149606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:42.149606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:42.149606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:42.149606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:42.149606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:42.150502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:42.150502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:42.150502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:42.150502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:42.154709 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 510/1036] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:42.155542 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf4e15-55cf-4bb7-9fd7-8737a19d2e33 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:42.174027 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb96c6-0e76-4311-be96-eb2f8ca6289c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:42.174915 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb96c6-0e76-4311-be96-eb2f8ca6289c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:42.175066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb96c6-0e76-4311-be96-eb2f8ca6289c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:42.175248 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb96c6-0e76-4311-be96-eb2f8ca6289c None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:42.175936 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb96c6-0e76-4311-be96-eb2f8ca6289c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:42.177336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb96c6-0e76-4311-be96-eb2f8ca6289c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:42.196356 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 527/1037] 10.222.0.94 () {56 vars in 1087 bytes} [Mon Jul 18 13:56:42 2022] GET /identity/v3/services => generated 1957 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 13:56:42.225501 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:42.270172 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf4e15-55cf-4bb7-9fd7-8737a19d2e33 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:42.311358 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ec95d-62ce-4693-896b-98bf46e5e832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:42.312069 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ec95d-62ce-4693-896b-98bf46e5e832 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:42.312357 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ec95d-62ce-4693-896b-98bf46e5e832 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:42.312636 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ec95d-62ce-4693-896b-98bf46e5e832 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:42.313431 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ec95d-62ce-4693-896b-98bf46e5e832 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:42.313745 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ec95d-62ce-4693-896b-98bf46e5e832 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:42.315146 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ec95d-62ce-4693-896b-98bf46e5e832 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:42.329041 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 511/1038] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:42.335025 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb96c6-0e76-4311-be96-eb2f8ca6289c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:42.383325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:42.446032 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb96c6-0e76-4311-be96-eb2f8ca6289c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:42.480307 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945a647-9b3e-4bfd-8c39-510a03435da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:42.481024 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945a647-9b3e-4bfd-8c39-510a03435da1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:42.481307 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945a647-9b3e-4bfd-8c39-510a03435da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:42.481580 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945a647-9b3e-4bfd-8c39-510a03435da1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:42.488211 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a945a647-9b3e-4bfd-8c39-510a03435da1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:42.489647 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a945a647-9b3e-4bfd-8c39-510a03435da1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:42.503594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a945a647-9b3e-4bfd-8c39-510a03435da1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:42.503594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:42.503594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:42.503594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:42.503594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:42.503594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:42.503594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:42.503594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:42.503594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:42.503594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:42.503594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:42.503594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:42.503594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:56:42.503594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:56:42.503594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:56:42.504683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:56:42.504683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:42.504683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:42.504683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:42.504683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:42.504683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:42.504683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:42.504683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:42.504683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:42.504683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:42.504683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:42.504683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:42.504683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:42.504683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:42.504683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:42.504683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:42.504683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:42.506360 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:56:42.506360 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:56:42.506360 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:56:42.506360 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:56:42.506360 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:56:42.506360 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:56:42.506360 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:42.506360 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:42.506360 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 528/1039] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:56:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:42.511981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ec95d-62ce-4693-896b-98bf46e5e832 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:42.612977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ec95d-62ce-4693-896b-98bf46e5e832 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:42.636972 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:42.650753 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf47c98-3956-42d1-8e6f-01dd851001ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:42.651426 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf47c98-3956-42d1-8e6f-01dd851001ba None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:42.651718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf47c98-3956-42d1-8e6f-01dd851001ba None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:42.652027 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf47c98-3956-42d1-8e6f-01dd851001ba None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:42.652823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf47c98-3956-42d1-8e6f-01dd851001ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:42.653140 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf47c98-3956-42d1-8e6f-01dd851001ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:42.654607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf47c98-3956-42d1-8e6f-01dd851001ba None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:42.667898 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 512/1040] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:56:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:42.673888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945a647-9b3e-4bfd-8c39-510a03435da1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:42.773618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:42.777264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945a647-9b3e-4bfd-8c39-510a03435da1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:42.811293 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12956853-1395-449d-bb68-67b6604d8cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:42.811953 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12956853-1395-449d-bb68-67b6604d8cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:42.812252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12956853-1395-449d-bb68-67b6604d8cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:42.812528 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12956853-1395-449d-bb68-67b6604d8cb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:42.813355 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12956853-1395-449d-bb68-67b6604d8cb9 None admin] RBAC: Adding query filter params (role.id=039f54fd5a574af19dad80e091f25108, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=ac717fcd9fc1499db12cf852d45572a8) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:42.813733 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12956853-1395-449d-bb68-67b6604d8cb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:42.815160 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12956853-1395-449d-bb68-67b6604d8cb9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:42.827995 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 529/1041] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:56:42 2022] GET /identity/v3/role_assignments?user.id=ac717fcd9fc1499db12cf852d45572a8&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=039f54fd5a574af19dad80e091f25108 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:43.197603 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e558ef-4427-4067-8744-a8c138abc13c None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:43.197958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e558ef-4427-4067-8744-a8c138abc13c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:43.198172 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e558ef-4427-4067-8744-a8c138abc13c None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:43.199681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 513/1042] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:43.204940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c81da-2bb0-4d7b-8e85-0a46915c72c2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:43.205240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c81da-2bb0-4d7b-8e85-0a46915c72c2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:43.205522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c81da-2bb0-4d7b-8e85-0a46915c72c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:43.222053 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f35c81da-2bb0-4d7b-8e85-0a46915c72c2 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:43.223231 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f35c81da-2bb0-4d7b-8e85-0a46915c72c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:43.294212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 530/1043] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:43.319030 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e558ef-4427-4067-8744-a8c138abc13c None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:43.425642 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e558ef-4427-4067-8744-a8c138abc13c None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:43.460208 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e9d64-f35e-4c98-8658-f3974a2d2b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:43.460935 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e9d64-f35e-4c98-8658-f3974a2d2b8b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:43.461224 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e9d64-f35e-4c98-8658-f3974a2d2b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:43.461506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e9d64-f35e-4c98-8658-f3974a2d2b8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:43.462711 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e9d64-f35e-4c98-8658-f3974a2d2b8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:43.464149 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e9d64-f35e-4c98-8658-f3974a2d2b8b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:43.470427 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 514/1044] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:43.471690 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c81da-2bb0-4d7b-8e85-0a46915c72c2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:43.563232 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:43.571127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c81da-2bb0-4d7b-8e85-0a46915c72c2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:43.604973 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d35f16c-e7cb-469c-ba63-8d3b031e6a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:43.605638 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35f16c-e7cb-469c-ba63-8d3b031e6a06 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:43.605926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35f16c-e7cb-469c-ba63-8d3b031e6a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:43.606203 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35f16c-e7cb-469c-ba63-8d3b031e6a06 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:43.606907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d35f16c-e7cb-469c-ba63-8d3b031e6a06 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:43.608385 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d35f16c-e7cb-469c-ba63-8d3b031e6a06 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:43.635979 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 531/1045] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Jul 18 13:56:43 2022] POST /identity/v3/users => generated 285 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 13:56:44.261059 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c1519-5e7c-4bbe-aee4-e3bd5abd68d6 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:44.261567 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c1519-5e7c-4bbe-aee4-e3bd5abd68d6 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:44.261567 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c1519-5e7c-4bbe-aee4-e3bd5abd68d6 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:44.263054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 515/1046] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:44.267960 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b02381-0239-4be6-8adc-00fb3d037cb1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:44.268266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b02381-0239-4be6-8adc-00fb3d037cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:44.268552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b02381-0239-4be6-8adc-00fb3d037cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:44.286995 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-15b02381-0239-4be6-8adc-00fb3d037cb1 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:44.288245 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-15b02381-0239-4be6-8adc-00fb3d037cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:44.359131 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 532/1047] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:44.382579 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c1519-5e7c-4bbe-aee4-e3bd5abd68d6 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:44.435912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:44.489169 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c1519-5e7c-4bbe-aee4-e3bd5abd68d6 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:44.524639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e92f16-b384-4f5c-8e4b-cf142bc0db15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:44.525707 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e92f16-b384-4f5c-8e4b-cf142bc0db15 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:44.526222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e92f16-b384-4f5c-8e4b-cf142bc0db15 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:44.526775 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e92f16-b384-4f5c-8e4b-cf142bc0db15 None admin] PATH_INFO: `/v3/services/image` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:44.527644 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e92f16-b384-4f5c-8e4b-cf142bc0db15 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:44.529261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e92f16-b384-4f5c-8e4b-cf142bc0db15 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:44.541757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-40e92f16-b384-4f5c-8e4b-cf142bc0db15 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 13:56:44.541757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:44.541757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:56:44.541757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:56:44.541757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:56:44.541757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:56:44.541757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:56:44.541757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:44.541757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 13:56:44.541757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:44.541757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:56:44.541757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:44.541757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:44.541757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:44.541757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:44.541757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:44.541757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:44.542767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:44.542767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:44.542767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:44.542767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:44.542767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:44.542767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:44.542767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:56:44.542767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:56:44.542767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:56:44.542767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:56:44.542767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:44.542767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:44.542767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:44.542767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:44.542767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:44.542767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:44.543640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:44.543640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:44.543640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:44.543640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:44.543640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:44.543640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:44.543640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:44.543640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:44.543640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:44.543640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:44.543640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:56:44.543640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:56:44.543640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 13:56:44.543640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:44.545413 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 516/1048] 10.222.0.94 () {56 vars in 1105 bytes} [Mon Jul 18 13:56:44 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:44.551121 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b02381-0239-4be6-8adc-00fb3d037cb1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:44.633267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:44.654839 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b02381-0239-4be6-8adc-00fb3d037cb1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:44.691485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81203119-c7a6-4f76-bab9-0477c05c59b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:44.692184 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81203119-c7a6-4f76-bab9-0477c05c59b4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:44.692477 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81203119-c7a6-4f76-bab9-0477c05c59b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:44.692761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81203119-c7a6-4f76-bab9-0477c05c59b4 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:44.693502 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81203119-c7a6-4f76-bab9-0477c05c59b4 None admin] RBAC: Adding query filter params (name=image) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:44.693815 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81203119-c7a6-4f76-bab9-0477c05c59b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:44.695188 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81203119-c7a6-4f76-bab9-0477c05c59b4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:44.704954 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 533/1049] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:44.706299 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e92f16-b384-4f5c-8e4b-cf142bc0db15 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:44.799109 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489144b-db76-4da1-a314-79dcd2ce94c1 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:44.799307 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489144b-db76-4da1-a314-79dcd2ce94c1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:44.799575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489144b-db76-4da1-a314-79dcd2ce94c1 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:44.800977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 534/1050] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:56:44.806478 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13184a4-2699-448c-b6c9-7c8e80feead8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:44.806769 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13184a4-2699-448c-b6c9-7c8e80feead8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:44.807049 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13184a4-2699-448c-b6c9-7c8e80feead8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:44.812741 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e92f16-b384-4f5c-8e4b-cf142bc0db15 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:44.826264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d13184a4-2699-448c-b6c9-7c8e80feead8 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:44.828030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d13184a4-2699-448c-b6c9-7c8e80feead8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:44.832468 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:44.858199 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eeff12-2f33-4b0c-85f4-e9f1d7142ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:44.859126 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eeff12-2f33-4b0c-85f4-e9f1d7142ade None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:44.859600 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eeff12-2f33-4b0c-85f4-e9f1d7142ade None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:44.859877 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eeff12-2f33-4b0c-85f4-e9f1d7142ade None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:44.860613 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eeff12-2f33-4b0c-85f4-e9f1d7142ade None admin] RBAC: Adding query filter params (type=image) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:44.861040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eeff12-2f33-4b0c-85f4-e9f1d7142ade None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:44.862504 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eeff12-2f33-4b0c-85f4-e9f1d7142ade None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:44.872521 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 517/1051] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:44 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:44.873820 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eeff12-2f33-4b0c-85f4-e9f1d7142ade None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:44.903717 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 535/1052] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 13:56:44.926502 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13184a4-2699-448c-b6c9-7c8e80feead8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:44.984198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eeff12-2f33-4b0c-85f4-e9f1d7142ade None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:45.004695 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:45.020783 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6962998-9858-4c13-ad9b-c1565b61fbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:45.022121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6962998-9858-4c13-ad9b-c1565b61fbe8 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:45.022345 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6962998-9858-4c13-ad9b-c1565b61fbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:45.022627 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6962998-9858-4c13-ad9b-c1565b61fbe8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:45.023367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6962998-9858-4c13-ad9b-c1565b61fbe8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:45.024767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6962998-9858-4c13-ad9b-c1565b61fbe8 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:45.031134 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13184a4-2699-448c-b6c9-7c8e80feead8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:45.045361 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 518/1053] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Jul 18 13:56:44 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:56:45.084163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730dd68-4a9c-45da-9b96-0569a2c102ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:45.084938 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730dd68-4a9c-45da-9b96-0569a2c102ef None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:45.085231 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730dd68-4a9c-45da-9b96-0569a2c102ef None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:45.085508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730dd68-4a9c-45da-9b96-0569a2c102ef None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:45.097967 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730dd68-4a9c-45da-9b96-0569a2c102ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:56:45.107066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5730dd68-4a9c-45da-9b96-0569a2c102ef None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 13:56:45.107066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:45.107066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:45.107066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:45.107066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:45.107066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:45.107066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:45.107066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:45.107066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:45.107066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:45.107066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:45.107066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:45.107066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:56:45.107066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:56:45.107066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:56:45.107066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:56:45.108106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:56:45.108106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:56:45.108106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:56:45.108106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:56:45.108106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:45.108106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:45.108106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:45.108106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:45.108106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:45.108106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:45.108106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:45.108106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:45.108106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:45.108106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:45.108106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:45.108106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:45.109051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:45.109051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:45.109051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:45.109051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:45.109051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:56:45.109051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:56:45.109051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:56:45.109051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:56:45.109051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:56:45.109051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:56:45.109051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 13:56:45.109051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:45.109051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 536/1054] 10.222.0.94 () {56 vars in 1120 bytes} [Mon Jul 18 13:56:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:45.114931 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6962998-9858-4c13-ad9b-c1565b61fbe8 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:45.153754 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:45.229048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6962998-9858-4c13-ad9b-c1565b61fbe8 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:45.266011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17784f13-8c39-46a9-9f6c-e6c4df05d7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:45.266786 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17784f13-8c39-46a9-9f6c-e6c4df05d7ae None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:45.266969 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17784f13-8c39-46a9-9f6c-e6c4df05d7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:45.267251 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17784f13-8c39-46a9-9f6c-e6c4df05d7ae None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:45.268035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17784f13-8c39-46a9-9f6c-e6c4df05d7ae None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:45.268351 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17784f13-8c39-46a9-9f6c-e6c4df05d7ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:45.269743 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17784f13-8c39-46a9-9f6c-e6c4df05d7ae None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:45.278362 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 519/1055] 10.222.0.94 () {56 vars in 1115 bytes} [Mon Jul 18 13:56:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:45.284137 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730dd68-4a9c-45da-9b96-0569a2c102ef None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:45.387556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730dd68-4a9c-45da-9b96-0569a2c102ef None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:45.416542 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:45.423624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c07c59-f1e7-4f9b-b6a5-bd569dc558d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:45.424445 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c07c59-f1e7-4f9b-b6a5-bd569dc558d5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:45.424734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c07c59-f1e7-4f9b-b6a5-bd569dc558d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:45.425014 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c07c59-f1e7-4f9b-b6a5-bd569dc558d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:45.436429 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c07c59-f1e7-4f9b-b6a5-bd569dc558d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:45.437946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c07c59-f1e7-4f9b-b6a5-bd569dc558d5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:45.455748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c9c07c59-f1e7-4f9b-b6a5-bd569dc558d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:45.455748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:45.455748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:45.455748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:45.455748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:45.455748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:45.455748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:45.455748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:45.455748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:45.455748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:45.455748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:45.455748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:45.455748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:45.455748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:45.455748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:45.456808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:45.456808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:45.456808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:45.456808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:45.456808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:45.456808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:45.456808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:45.456808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:45.456808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:45.456808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:45.456808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:45.456808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:45.456808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:45.456808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:45.456808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:45.456808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:45.456808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:45.457741 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:45.457741 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:45.457741 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:45.457741 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:45.457741 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 537/1056] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:45.464574 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17784f13-8c39-46a9-9f6c-e6c4df05d7ae None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:45.563430 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:45.571007 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17784f13-8c39-46a9-9f6c-e6c4df05d7ae None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:45.605182 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e86294-24d1-40e1-beeb-f4935f19089b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:45.605851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e86294-24d1-40e1-beeb-f4935f19089b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:45.606138 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e86294-24d1-40e1-beeb-f4935f19089b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:45.606417 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e86294-24d1-40e1-beeb-f4935f19089b None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:45.607143 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e86294-24d1-40e1-beeb-f4935f19089b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:45.607451 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e86294-24d1-40e1-beeb-f4935f19089b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:45.608857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e86294-24d1-40e1-beeb-f4935f19089b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:45.624026 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 520/1057] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:45.626906 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c07c59-f1e7-4f9b-b6a5-bd569dc558d5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:45.701040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d1524-669b-48f7-b3b0-74a92ff1cb51 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:45.701286 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d1524-669b-48f7-b3b0-74a92ff1cb51 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:45.701516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d1524-669b-48f7-b3b0-74a92ff1cb51 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:45.702883 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 521/1058] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:45.708205 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72ca52-eb81-4b42-920a-6d7d62d19d60 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:45.708593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72ca52-eb81-4b42-920a-6d7d62d19d60 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:45.708955 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72ca52-eb81-4b42-920a-6d7d62d19d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:45.725526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ed72ca52-eb81-4b42-920a-6d7d62d19d60 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:45.726706 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ed72ca52-eb81-4b42-920a-6d7d62d19d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:45.734655 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c07c59-f1e7-4f9b-b6a5-bd569dc558d5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:45.771906 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c399a-d623-4dd4-83a9-86ef9b22a812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:45.773101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c399a-d623-4dd4-83a9-86ef9b22a812 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:45.773391 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c399a-d623-4dd4-83a9-86ef9b22a812 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:45.773670 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c399a-d623-4dd4-83a9-86ef9b22a812 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:45.781259 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c399a-d623-4dd4-83a9-86ef9b22a812 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:45.783230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c399a-d623-4dd4-83a9-86ef9b22a812 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:45.797597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3d8c399a-d623-4dd4-83a9-86ef9b22a812 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 13:56:45.797597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:45.797597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:45.797597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:45.797597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:45.797597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:45.797597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:45.797597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:45.797597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:45.797597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:45.797597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:45.797597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:45.797597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:56:45.797597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:56:45.797597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:56:45.797597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:56:45.798575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:45.798575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:45.798575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:56:45.798575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:45.798575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:56:45.798575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:45.798575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:45.798575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:45.798575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:45.798575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:45.798575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:45.798575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:45.798575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:45.798575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:45.798575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:45.798575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:45.798575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:45.798575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:45.799553 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:45.799553 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:45.799553 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:56:45.799553 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:56:45.799553 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:56:45.799553 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:56:45.799553 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:56:45.799553 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:56:45.799553 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 13:56:45.799553 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:45.799553 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 538/1059] 10.222.0.94 () {56 vars in 1093 bytes} [Mon Jul 18 13:56:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:45.804812 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c399a-d623-4dd4-83a9-86ef9b22a812 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:45.807979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 522/1060] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 13:56:45.828225 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72ca52-eb81-4b42-920a-6d7d62d19d60 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:45.852923 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:45.914793 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c399a-d623-4dd4-83a9-86ef9b22a812 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:45.918059 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:45.934666 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72ca52-eb81-4b42-920a-6d7d62d19d60 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:45.957582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853268e-ea2d-466f-95a3-3cff0d48be1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:45.958202 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853268e-ea2d-466f-95a3-3cff0d48be1c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:45.959012 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853268e-ea2d-466f-95a3-3cff0d48be1c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:45.959296 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853268e-ea2d-466f-95a3-3cff0d48be1c None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:45.960174 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853268e-ea2d-466f-95a3-3cff0d48be1c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:45.960489 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853268e-ea2d-466f-95a3-3cff0d48be1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:45.962014 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853268e-ea2d-466f-95a3-3cff0d48be1c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:45.979570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638405a-6b8d-4ff2-9912-ab635c20fd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:45.980514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638405a-6b8d-4ff2-9912-ab635c20fd1c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:45.980868 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638405a-6b8d-4ff2-9912-ab635c20fd1c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:45.981215 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638405a-6b8d-4ff2-9912-ab635c20fd1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:45.993120 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 539/1061] 10.222.0.94 () {56 vars in 1133 bytes} [Mon Jul 18 13:56:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:45.998723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853268e-ea2d-466f-95a3-3cff0d48be1c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:46.000297 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d638405a-6b8d-4ff2-9912-ab635c20fd1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:56:46.007828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-d638405a-6b8d-4ff2-9912-ab635c20fd1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:56:46.007828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:46.007828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:46.007828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:46.007828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:46.007828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:46.007828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:46.007828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:46.007828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:46.007828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:46.007828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:46.007828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:46.007828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:56:46.007828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:56:46.007828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:56:46.007828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:56:46.008915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:56:46.008915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:56:46.008915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:56:46.008915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:56:46.008915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:46.008915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:46.008915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:46.008915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:46.008915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:46.008915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:46.008915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:46.008915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:46.008915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:46.008915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:46.008915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:46.008915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:46.009771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:46.009771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:46.009771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:46.009771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:46.009771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:56:46.009771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:56:46.009771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:56:46.009771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:56:46.009771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:56:46.009771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:56:46.009771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:56:46.009771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:46.009771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 523/1062] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:56:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:46.014871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638405a-6b8d-4ff2-9912-ab635c20fd1c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:46.108965 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:46.112241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853268e-ea2d-466f-95a3-3cff0d48be1c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:46.117495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638405a-6b8d-4ff2-9912-ab635c20fd1c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:46.151629 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4183a-c6be-4911-bd64-183d4acbfa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:46.152457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4183a-c6be-4911-bd64-183d4acbfa4f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:46.152744 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4183a-c6be-4911-bd64-183d4acbfa4f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:46.153021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4183a-c6be-4911-bd64-183d4acbfa4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:46.153213 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b317b0-cb40-4dec-8a5d-45bcd318fb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:46.153880 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b317b0-cb40-4dec-8a5d-45bcd318fb4b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:46.154166 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b317b0-cb40-4dec-8a5d-45bcd318fb4b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:46.154441 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b317b0-cb40-4dec-8a5d-45bcd318fb4b None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:46.155185 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b317b0-cb40-4dec-8a5d-45bcd318fb4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:46.155490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b317b0-cb40-4dec-8a5d-45bcd318fb4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:46.156880 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b317b0-cb40-4dec-8a5d-45bcd318fb4b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:46.163742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c4183a-c6be-4911-bd64-183d4acbfa4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:46.165469 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c4183a-c6be-4911-bd64-183d4acbfa4f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:46.165578 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 524/1063] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:56:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:46.171155 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b317b0-cb40-4dec-8a5d-45bcd318fb4b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:46.183853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:46.184522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-24c4183a-c6be-4911-bd64-183d4acbfa4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:46.184522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:46.184522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:46.184522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:46.184522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:46.184522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:46.184522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:46.184522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:46.184522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:46.184522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:46.184522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:46.184522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:46.184522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:46.184522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:46.184522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:46.185402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:46.185402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:46.185402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:46.185402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:46.185402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:46.185402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:46.185402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:46.185402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:46.185402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:46.185402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:46.185402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:46.185402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:46.185402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:46.185402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:46.185402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:46.185402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:46.185402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:46.186318 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:46.186318 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:46.186318 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:46.186318 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:46.186318 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 540/1064] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:46.191674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4183a-c6be-4911-bd64-183d4acbfa4f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:46.281150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b317b0-cb40-4dec-8a5d-45bcd318fb4b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:46.291837 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4183a-c6be-4911-bd64-183d4acbfa4f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:46.316120 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbb670-ede4-49aa-b1b5-66e2598fb54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:46.317109 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbb670-ede4-49aa-b1b5-66e2598fb54e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:46.317514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbb670-ede4-49aa-b1b5-66e2598fb54e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:46.317793 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbb670-ede4-49aa-b1b5-66e2598fb54e None admin] PATH_INFO: `/v3/users/f89df19d05e54b5c8df015ebd9e22286` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:46.325726 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc81df5-63a5-4859-91db-7de117b62523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:46.326408 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc81df5-63a5-4859-91db-7de117b62523 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:46.326706 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc81df5-63a5-4859-91db-7de117b62523 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:46.326990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc81df5-63a5-4859-91db-7de117b62523 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:46.327739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc81df5-63a5-4859-91db-7de117b62523 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:46.328088 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc81df5-63a5-4859-91db-7de117b62523 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:46.329756 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc81df5-63a5-4859-91db-7de117b62523 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:46.334837 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fbb670-ede4-49aa-b1b5-66e2598fb54e None admin] RBAC: Authorizing `identity:get_user(user_id=f89df19d05e54b5c8df015ebd9e22286)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:46.336579 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fbb670-ede4-49aa-b1b5-66e2598fb54e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:46.339046 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 525/1065] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:56:46 2022] GET /identity/v3/users/f89df19d05e54b5c8df015ebd9e22286 => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:46.344286 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbb670-ede4-49aa-b1b5-66e2598fb54e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:46.346846 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 541/1066] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:46.352592 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc81df5-63a5-4859-91db-7de117b62523 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:46.367664 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:46.418055 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:46.458970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc81df5-63a5-4859-91db-7de117b62523 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:46.462179 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbb670-ede4-49aa-b1b5-66e2598fb54e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:46.494371 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68a018-050f-4f30-9102-fd720435da11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:46.495830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68a018-050f-4f30-9102-fd720435da11 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:46.496311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68a018-050f-4f30-9102-fd720435da11 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:46.496737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68a018-050f-4f30-9102-fd720435da11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:46.500294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc9209-3de9-497e-9ef1-f1ab946944d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:46.500944 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc9209-3de9-497e-9ef1-f1ab946944d4 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:46.501229 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc9209-3de9-497e-9ef1-f1ab946944d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:46.501510 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc9209-3de9-497e-9ef1-f1ab946944d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:46.504427 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe68a018-050f-4f30-9102-fd720435da11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:46.506725 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe68a018-050f-4f30-9102-fd720435da11 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:46.521264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-fe68a018-050f-4f30-9102-fd720435da11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:46.521264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:46.521264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:46.521264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:46.521264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:46.521264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:46.521264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:46.521264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:46.521264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:46.521264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:46.521264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:46.521264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:46.521264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:56:46.521264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:56:46.521264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:56:46.522339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:56:46.522339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:46.522339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:46.522339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:46.522339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:46.522339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:46.522339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:46.522339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:46.522339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:46.522339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:46.522339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:46.522339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:46.522339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:46.522339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:46.522339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:46.522339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:46.522339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:46.523445 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:56:46.523445 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:56:46.523445 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:56:46.523445 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:56:46.523445 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:56:46.523445 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:56:46.523445 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:46.523445 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:46.525276 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 542/1067] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:56:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:46.531307 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68a018-050f-4f30-9102-fd720435da11 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:46.573510 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc9209-3de9-497e-9ef1-f1ab946944d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:46.575100 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc9209-3de9-497e-9ef1-f1ab946944d4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:46.606883 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:46.639290 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68a018-050f-4f30-9102-fd720435da11 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:46.645723 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:46.675188 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35488b8b-03eb-426c-87f5-854098972ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:46.675854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35488b8b-03eb-426c-87f5-854098972ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:46.676174 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35488b8b-03eb-426c-87f5-854098972ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:46.676672 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35488b8b-03eb-426c-87f5-854098972ad9 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:46.677771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35488b8b-03eb-426c-87f5-854098972ad9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:46.678117 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35488b8b-03eb-426c-87f5-854098972ad9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:46.679530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35488b8b-03eb-426c-87f5-854098972ad9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:46.684052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 526/1068] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:56:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 13:56:46.689564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc9209-3de9-497e-9ef1-f1ab946944d4 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:46.691778 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 543/1069] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:56:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:46.697398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35488b8b-03eb-426c-87f5-854098972ad9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:46.799462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc9209-3de9-497e-9ef1-f1ab946944d4 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:46.802548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35488b8b-03eb-426c-87f5-854098972ad9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:46.835812 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:46.836183 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553086e-b4fe-4197-993d-12761ffdc555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:46.837076 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553086e-b4fe-4197-993d-12761ffdc555 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:46.837411 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553086e-b4fe-4197-993d-12761ffdc555 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:46.837717 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553086e-b4fe-4197-993d-12761ffdc555 None admin] PATH_INFO: `/v3/projects/70aab481bb27420cbd2f531a01d3e08c` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:46.839178 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553086e-b4fe-4197-993d-12761ffdc555 None admin] RBAC: Authorizing `identity:get_project(project_id=70aab481bb27420cbd2f531a01d3e08c)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:46.840790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553086e-b4fe-4197-993d-12761ffdc555 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:46.842138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a0e1c-706a-493b-8f08-0af6e2dec844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:46.842640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 527/1070] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:56:46 2022] GET /identity/v3/projects/70aab481bb27420cbd2f531a01d3e08c => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:46.843461 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a0e1c-706a-493b-8f08-0af6e2dec844 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:46.843754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a0e1c-706a-493b-8f08-0af6e2dec844 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:46.844062 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a0e1c-706a-493b-8f08-0af6e2dec844 None admin] PATH_INFO: `/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/ac717fcd9fc1499db12cf852d45572a8/roles/039f54fd5a574af19dad80e091f25108` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:46.848598 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553086e-b4fe-4197-993d-12761ffdc555 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:46.849267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a0e1c-706a-493b-8f08-0af6e2dec844 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec94946acf91486ba3f5b4fe834b8076, user_id=ac717fcd9fc1499db12cf852d45572a8, role_id=039f54fd5a574af19dad80e091f25108)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:46.850694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a0e1c-706a-493b-8f08-0af6e2dec844 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:46.864902 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 544/1071] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:56:46 2022] PUT /identity/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/ac717fcd9fc1499db12cf852d45572a8/roles/039f54fd5a574af19dad80e091f25108 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 13:56:46.891158 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:46.973189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553086e-b4fe-4197-993d-12761ffdc555 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:47.010881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09b618-61d4-42c1-843a-feb0fa45623f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:47.011551 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09b618-61d4-42c1-843a-feb0fa45623f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:47.011844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09b618-61d4-42c1-843a-feb0fa45623f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:47.012156 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09b618-61d4-42c1-843a-feb0fa45623f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:47.013001 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09b618-61d4-42c1-843a-feb0fa45623f None admin] RBAC: Adding query filter params (role.id=c60a99de086f45c6820f7872a6b43feb, scope.project.id=70aab481bb27420cbd2f531a01d3e08c, user.id=f89df19d05e54b5c8df015ebd9e22286) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:47.013368 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09b618-61d4-42c1-843a-feb0fa45623f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:47.014820 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09b618-61d4-42c1-843a-feb0fa45623f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:47.021084 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 528/1072] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:56:46 2022] GET /identity/v3/role_assignments?user.id=f89df19d05e54b5c8df015ebd9e22286&scope.project.id=70aab481bb27420cbd2f531a01d3e08c&role.id=c60a99de086f45c6820f7872a6b43feb => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:47.096368 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083beb96-7fb6-4e90-ae92-e0e2fe937732 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:47.096622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083beb96-7fb6-4e90-ae92-e0e2fe937732 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:47.097012 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083beb96-7fb6-4e90-ae92-e0e2fe937732 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:47.098639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 545/1073] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:47.104652 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a161cfe-7db1-43ee-8d2d-f275f7fe0d5e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:47.104943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a161cfe-7db1-43ee-8d2d-f275f7fe0d5e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:47.105222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a161cfe-7db1-43ee-8d2d-f275f7fe0d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:47.123181 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-6a161cfe-7db1-43ee-8d2d-f275f7fe0d5e None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:47.124452 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6a161cfe-7db1-43ee-8d2d-f275f7fe0d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:47.198379 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 529/1074] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:47.220718 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083beb96-7fb6-4e90-ae92-e0e2fe937732 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:47.327874 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083beb96-7fb6-4e90-ae92-e0e2fe937732 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:47.363147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6afaa-c7a9-470f-aadd-298ed4c79098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:47.363750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6afaa-c7a9-470f-aadd-298ed4c79098 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:47.364067 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6afaa-c7a9-470f-aadd-298ed4c79098 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:47.364351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6afaa-c7a9-470f-aadd-298ed4c79098 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:47.427775 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:47.439929 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6afaa-c7a9-470f-aadd-298ed4c79098 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:47.441426 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6afaa-c7a9-470f-aadd-298ed4c79098 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:47.541903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 546/1075] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:56:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 13:56:47.547700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a161cfe-7db1-43ee-8d2d-f275f7fe0d5e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:47.575596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:47.654869 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a161cfe-7db1-43ee-8d2d-f275f7fe0d5e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:47.691436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b69cc-1c43-459c-af6c-0fecb2797ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:47.692268 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b69cc-1c43-459c-af6c-0fecb2797ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:47.692562 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b69cc-1c43-459c-af6c-0fecb2797ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:47.693362 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b69cc-1c43-459c-af6c-0fecb2797ee0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:47.703984 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b69cc-1c43-459c-af6c-0fecb2797ee0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:47.712562 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b69cc-1c43-459c-af6c-0fecb2797ee0 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:47.736871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-b08b69cc-1c43-459c-af6c-0fecb2797ee0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:47.736871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:47.736871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:47.736871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:47.736871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:47.736871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:47.736871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:47.736871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:47.736871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:47.736871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:47.736871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:47.736871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:47.736871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:47.736871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:47.736871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:47.737809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:47.737809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:47.737809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:47.737809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:47.737809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:47.737809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:47.737809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:47.737809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:47.737809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:47.737809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:47.737809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:47.737809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:47.737809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:47.737809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:47.737809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:47.737809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:47.737809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:47.742566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:47.742566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:47.742566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:47.742566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:47.742566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 530/1076] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:47.745855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6afaa-c7a9-470f-aadd-298ed4c79098 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:47.836577 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:47.846534 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6afaa-c7a9-470f-aadd-298ed4c79098 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:47.880007 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7064049-18e3-4416-8795-4a314281656b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:47.881200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7064049-18e3-4416-8795-4a314281656b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:47.881492 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7064049-18e3-4416-8795-4a314281656b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:47.881772 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7064049-18e3-4416-8795-4a314281656b None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:47.882496 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7064049-18e3-4416-8795-4a314281656b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:47.882807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7064049-18e3-4416-8795-4a314281656b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:47.884202 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7064049-18e3-4416-8795-4a314281656b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:47.903974 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 547/1077] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:48.888054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda4a78-484b-4316-a2d8-c3fc0a147b3c None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:48.888582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda4a78-484b-4316-a2d8-c3fc0a147b3c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:48.888699 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda4a78-484b-4316-a2d8-c3fc0a147b3c None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:48.889997 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 531/1078] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:48.895270 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0179b-cc79-4604-9e80-f7c5d61dbf1d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:48.895692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0179b-cc79-4604-9e80-f7c5d61dbf1d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:48.896346 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0179b-cc79-4604-9e80-f7c5d61dbf1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:48.913384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7ef0179b-cc79-4604-9e80-f7c5d61dbf1d None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:48.914576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7ef0179b-cc79-4604-9e80-f7c5d61dbf1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:48.982487 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 548/1079] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:48.999457 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b2e09-f45a-4f29-a46e-9e987ed7353a None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:48.999748 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b2e09-f45a-4f29-a46e-9e987ed7353a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:49.000056 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b2e09-f45a-4f29-a46e-9e987ed7353a None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:49.001408 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 532/1080] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:49.005115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0179b-cc79-4604-9e80-f7c5d61dbf1d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:49.006184 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5eb5a8-5231-4bb0-acf1-5b9b8aefe029 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:49.006479 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5eb5a8-5231-4bb0-acf1-5b9b8aefe029 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:49.006765 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5eb5a8-5231-4bb0-acf1-5b9b8aefe029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:49.023527 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-3b5eb5a8-5231-4bb0-acf1-5b9b8aefe029 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:49.024740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3b5eb5a8-5231-4bb0-acf1-5b9b8aefe029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:49.087106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:49.091495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 533/1081] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 13:56:49.105424 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0179b-cc79-4604-9e80-f7c5d61dbf1d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:49.113406 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5eb5a8-5231-4bb0-acf1-5b9b8aefe029 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:49.124708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:49.140960 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8de40a-783c-47fa-8f16-6066e5e12c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:49.141716 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8de40a-783c-47fa-8f16-6066e5e12c17 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:49.142003 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8de40a-783c-47fa-8f16-6066e5e12c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:49.142284 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8de40a-783c-47fa-8f16-6066e5e12c17 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:49.152964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8de40a-783c-47fa-8f16-6066e5e12c17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:56:49.161085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-4c8de40a-783c-47fa-8f16-6066e5e12c17 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 13:56:49.161085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:49.161085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:49.161085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:49.161085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:49.161085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:49.161085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:49.161085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:49.161085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:49.161085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:49.161085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:49.161085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:49.161085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:56:49.161085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:56:49.161085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:56:49.161085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:56:49.162030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:56:49.162030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:56:49.162030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:56:49.162030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:56:49.162030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:49.162030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:49.162030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:49.162030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:49.162030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:49.162030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:49.162030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:49.162030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:49.162030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:49.162030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:49.162030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:49.162030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:49.162887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:49.162887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:49.162887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:49.162887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:49.162887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:56:49.162887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:56:49.162887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:56:49.162887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:56:49.162887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:56:49.162887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:56:49.162887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 13:56:49.162887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:49.162887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 549/1082] 10.222.0.94 () {56 vars in 1120 bytes} [Mon Jul 18 13:56:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:49.168182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8de40a-783c-47fa-8f16-6066e5e12c17 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:49.211636 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5eb5a8-5231-4bb0-acf1-5b9b8aefe029 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:49.244597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a8cca-2aef-4e60-b793-876d7a8c1c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:49.245391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a8cca-2aef-4e60-b793-876d7a8c1c27 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:49.245682 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a8cca-2aef-4e60-b793-876d7a8c1c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:49.245956 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a8cca-2aef-4e60-b793-876d7a8c1c27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:49.256440 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5a8cca-2aef-4e60-b793-876d7a8c1c27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:56:49.263986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-7b5a8cca-2aef-4e60-b793-876d7a8c1c27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:56:49.263986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:49.263986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:49.263986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:49.263986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:49.263986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:49.263986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:49.263986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:49.263986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:49.263986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:49.263986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:49.263986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:49.263986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:56:49.263986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:56:49.263986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:56:49.263986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:56:49.264926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:56:49.264926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:56:49.264926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:56:49.264926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:56:49.264926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:49.264926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:49.264926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:49.264926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:49.264926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:49.264926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:49.264926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:49.264926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:49.264926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:49.264926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:49.264926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:49.264926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:49.265803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:49.265803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:49.265803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:49.265803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:49.265803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:56:49.265803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:56:49.265803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:56:49.265803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:56:49.265803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:56:49.265803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:56:49.265803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:56:49.265803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:49.266456 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8de40a-783c-47fa-8f16-6066e5e12c17 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:49.268240 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 534/1083] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:56:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:49.271037 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a8cca-2aef-4e60-b793-876d7a8c1c27 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:49.300027 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63bcbb5-0829-48de-8e1a-c6ea38c49062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:49.300847 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bcbb5-0829-48de-8e1a-c6ea38c49062 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:49.301433 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bcbb5-0829-48de-8e1a-c6ea38c49062 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:49.301736 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bcbb5-0829-48de-8e1a-c6ea38c49062 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:49.303059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63bcbb5-0829-48de-8e1a-c6ea38c49062 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:49.303375 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63bcbb5-0829-48de-8e1a-c6ea38c49062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:49.304771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63bcbb5-0829-48de-8e1a-c6ea38c49062 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:49.317117 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 550/1084] 10.222.0.94 () {56 vars in 1115 bytes} [Mon Jul 18 13:56:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:49.318112 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63bcbb5-0829-48de-8e1a-c6ea38c49062 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:49.321959 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:49.339746 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:49.371607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a8cca-2aef-4e60-b793-876d7a8c1c27 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:49.410944 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8416c3d-3b14-4165-b000-cd64151166b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:49.411547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8416c3d-3b14-4165-b000-cd64151166b9 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:49.411836 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8416c3d-3b14-4165-b000-cd64151166b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:49.412142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8416c3d-3b14-4165-b000-cd64151166b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:49.412905 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8416c3d-3b14-4165-b000-cd64151166b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:49.413216 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8416c3d-3b14-4165-b000-cd64151166b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:49.414708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8416c3d-3b14-4165-b000-cd64151166b9 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:49.418324 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63bcbb5-0829-48de-8e1a-c6ea38c49062 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:49.423335 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 535/1085] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:56:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:49.429686 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8416c3d-3b14-4165-b000-cd64151166b9 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:49.452394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e8c42-6e44-4b15-96f6-3546fe5e4d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:49.453193 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e8c42-6e44-4b15-96f6-3546fe5e4d36 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:49.453750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e8c42-6e44-4b15-96f6-3546fe5e4d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:49.454032 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e8c42-6e44-4b15-96f6-3546fe5e4d36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:49.464493 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e8c42-6e44-4b15-96f6-3546fe5e4d36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:49.465955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e8c42-6e44-4b15-96f6-3546fe5e4d36 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:49.482624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-1d3e8c42-6e44-4b15-96f6-3546fe5e4d36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:49.482624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:49.482624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:49.482624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:49.482624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:49.482624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:49.482624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:49.482624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:49.482624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:49.482624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:49.482624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:49.482624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:49.482624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:49.482624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:49.482624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:49.483555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:49.483555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:49.483555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:49.483555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:49.483555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:49.483555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:49.483555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:49.483555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:49.483555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:49.483555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:49.483555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:49.483555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:49.483555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:49.483555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:49.483555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:49.483555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:49.483555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:49.484607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:49.484607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:49.484607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:49.484607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:49.484607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 551/1086] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:49.489359 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e8c42-6e44-4b15-96f6-3546fe5e4d36 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:49.528839 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8416c3d-3b14-4165-b000-cd64151166b9 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:49.543834 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:49.557658 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:49.563123 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc5b55-dc50-4d99-b191-9e0e658db4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:49.564037 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc5b55-dc50-4d99-b191-9e0e658db4dc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:49.564327 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc5b55-dc50-4d99-b191-9e0e658db4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:49.564602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc5b55-dc50-4d99-b191-9e0e658db4dc None admin] PATH_INFO: `/v3/users/f89df19d05e54b5c8df015ebd9e22286` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:49.566091 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc5b55-dc50-4d99-b191-9e0e658db4dc None admin] RBAC: Authorizing `identity:get_user(user_id=f89df19d05e54b5c8df015ebd9e22286)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:49.567516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc5b55-dc50-4d99-b191-9e0e658db4dc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:49.569795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 536/1087] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:56:49 2022] GET /identity/v3/users/f89df19d05e54b5c8df015ebd9e22286 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:49.575449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc5b55-dc50-4d99-b191-9e0e658db4dc None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:49.589929 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e8c42-6e44-4b15-96f6-3546fe5e4d36 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:49.625102 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdb782-9cd0-4f02-ba20-481b6169bccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:49.625770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdb782-9cd0-4f02-ba20-481b6169bccb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:49.626064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdb782-9cd0-4f02-ba20-481b6169bccb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:49.626350 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdb782-9cd0-4f02-ba20-481b6169bccb None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:49.627073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfdb782-9cd0-4f02-ba20-481b6169bccb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:49.627385 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfdb782-9cd0-4f02-ba20-481b6169bccb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:49.628791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfdb782-9cd0-4f02-ba20-481b6169bccb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:49.640475 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 552/1088] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:49.646441 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdb782-9cd0-4f02-ba20-481b6169bccb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:49.672879 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc5b55-dc50-4d99-b191-9e0e658db4dc None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:49.706876 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc116dc8-9d01-4c53-a6b5-edd301c852bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:49.707725 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc116dc8-9d01-4c53-a6b5-edd301c852bc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:49.708048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc116dc8-9d01-4c53-a6b5-edd301c852bc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:49.708822 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc116dc8-9d01-4c53-a6b5-edd301c852bc None admin] PATH_INFO: `/v3/projects/70aab481bb27420cbd2f531a01d3e08c` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:49.710008 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc116dc8-9d01-4c53-a6b5-edd301c852bc None admin] RBAC: Authorizing `identity:get_project(project_id=70aab481bb27420cbd2f531a01d3e08c)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:49.711444 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc116dc8-9d01-4c53-a6b5-edd301c852bc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:49.713170 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 537/1089] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:56:49 2022] GET /identity/v3/projects/70aab481bb27420cbd2f531a01d3e08c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:49.719129 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc116dc8-9d01-4c53-a6b5-edd301c852bc None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:49.743560 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdb782-9cd0-4f02-ba20-481b6169bccb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:49.759473 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:49.778777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb23dee-3e4f-4844-86e3-cdf4dc66b114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:49.779584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb23dee-3e4f-4844-86e3-cdf4dc66b114 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:49.779872 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb23dee-3e4f-4844-86e3-cdf4dc66b114 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:49.780236 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb23dee-3e4f-4844-86e3-cdf4dc66b114 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:49.784065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:49.787290 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb23dee-3e4f-4844-86e3-cdf4dc66b114 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:49.788747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb23dee-3e4f-4844-86e3-cdf4dc66b114 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:49.802752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3cb23dee-3e4f-4844-86e3-cdf4dc66b114 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 13:56:49.802752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:49.802752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:49.802752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:49.802752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:49.802752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:49.802752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:49.802752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:49.802752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:49.802752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:49.802752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:49.802752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:49.802752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:56:49.802752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:56:49.802752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:56:49.802752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:56:49.803711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:49.803711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:49.803711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:56:49.803711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:49.803711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:56:49.803711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:56:49.803711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:49.803711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:49.803711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:49.803711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:49.803711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:49.803711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:49.803711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:49.803711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:49.803711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:49.803711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:49.803711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:49.803711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:49.805266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:49.805266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:49.805266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:56:49.805266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:56:49.805266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:56:49.805266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:56:49.805266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:56:49.805266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:56:49.805266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 13:56:49.805266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:49.805266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 553/1090] 10.222.0.94 () {56 vars in 1093 bytes} [Mon Jul 18 13:56:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:49.809510 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb23dee-3e4f-4844-86e3-cdf4dc66b114 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:49.819634 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc116dc8-9d01-4c53-a6b5-edd301c852bc None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:49.853151 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f788aa-aab2-4d56-98ff-92c6f734704d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:49.854032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f788aa-aab2-4d56-98ff-92c6f734704d None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:49.854320 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f788aa-aab2-4d56-98ff-92c6f734704d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:49.854606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f788aa-aab2-4d56-98ff-92c6f734704d None admin] PATH_INFO: `/v3/projects/70aab481bb27420cbd2f531a01d3e08c/users/f89df19d05e54b5c8df015ebd9e22286/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:49.856604 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f788aa-aab2-4d56-98ff-92c6f734704d None admin] RBAC: Authorizing `identity:create_grant(project_id=70aab481bb27420cbd2f531a01d3e08c, user_id=f89df19d05e54b5c8df015ebd9e22286, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:49.858040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f788aa-aab2-4d56-98ff-92c6f734704d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:49.871433 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 538/1091] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:56:49 2022] PUT /identity/v3/projects/70aab481bb27420cbd2f531a01d3e08c/users/f89df19d05e54b5c8df015ebd9e22286/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:56:49.911875 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb23dee-3e4f-4844-86e3-cdf4dc66b114 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:49.954271 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95088044-2eea-42a0-ae8b-e8ae9891667a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:49.954987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95088044-2eea-42a0-ae8b-e8ae9891667a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:49.955279 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95088044-2eea-42a0-ae8b-e8ae9891667a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:49.955561 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95088044-2eea-42a0-ae8b-e8ae9891667a None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:49.956464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95088044-2eea-42a0-ae8b-e8ae9891667a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:49.956784 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95088044-2eea-42a0-ae8b-e8ae9891667a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:49.958291 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95088044-2eea-42a0-ae8b-e8ae9891667a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:49.982463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 554/1092] 10.222.0.94 () {56 vars in 1133 bytes} [Mon Jul 18 13:56:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:49.983735 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f788aa-aab2-4d56-98ff-92c6f734704d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:50.086339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f788aa-aab2-4d56-98ff-92c6f734704d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:50.111735 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:50.120967 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28067856-a937-42c7-868a-f90c19fde6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:50.121619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28067856-a937-42c7-868a-f90c19fde6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:50.121901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28067856-a937-42c7-868a-f90c19fde6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:50.122179 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28067856-a937-42c7-868a-f90c19fde6b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:50.186792 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28067856-a937-42c7-868a-f90c19fde6b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:50.188267 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28067856-a937-42c7-868a-f90c19fde6b0 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:50.282212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 539/1093] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:56:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 13:56:50.288545 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95088044-2eea-42a0-ae8b-e8ae9891667a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:50.321570 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:50.386738 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95088044-2eea-42a0-ae8b-e8ae9891667a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:50.419732 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06158310-1b93-487d-808c-3885dd0e603f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:50.420537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06158310-1b93-487d-808c-3885dd0e603f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:50.420820 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06158310-1b93-487d-808c-3885dd0e603f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:50.421094 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06158310-1b93-487d-808c-3885dd0e603f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:50.431363 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06158310-1b93-487d-808c-3885dd0e603f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:50.432817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06158310-1b93-487d-808c-3885dd0e603f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:50.449622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-06158310-1b93-487d-808c-3885dd0e603f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:50.449622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:50.449622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:50.449622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:50.449622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:50.449622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:50.449622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:50.449622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:50.449622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:50.449622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:50.449622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:50.449622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:50.449622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:56:50.449622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:56:50.449622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:56:50.450641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:56:50.450641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:50.450641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:50.450641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:50.450641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:50.450641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:50.450641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:50.450641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:50.450641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:50.450641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:50.450641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:50.450641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:50.450641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:50.450641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:50.450641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:50.450641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:50.450641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:50.451597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:56:50.451597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:56:50.451597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:56:50.451597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:50.451597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 555/1094] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:50.456667 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28067856-a937-42c7-868a-f90c19fde6b0 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:50.483474 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:50.554748 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28067856-a937-42c7-868a-f90c19fde6b0 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:50.588022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e066c-811b-4036-85d3-1cd7b57f3c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:50.588678 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e066c-811b-4036-85d3-1cd7b57f3c7d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:50.588962 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e066c-811b-4036-85d3-1cd7b57f3c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:50.589237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e066c-811b-4036-85d3-1cd7b57f3c7d None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:50.589956 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e066c-811b-4036-85d3-1cd7b57f3c7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:50.590260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e066c-811b-4036-85d3-1cd7b57f3c7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:50.591622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e066c-811b-4036-85d3-1cd7b57f3c7d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:50.603176 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 540/1095] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:56:50.608785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06158310-1b93-487d-808c-3885dd0e603f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:50.691297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:50.706386 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06158310-1b93-487d-808c-3885dd0e603f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:50.738609 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60133ba-a7e2-4894-9fc4-733e77fd2093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:50.739386 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60133ba-a7e2-4894-9fc4-733e77fd2093 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:50.740247 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60133ba-a7e2-4894-9fc4-733e77fd2093 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:50.740523 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60133ba-a7e2-4894-9fc4-733e77fd2093 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:50.746925 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60133ba-a7e2-4894-9fc4-733e77fd2093 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:50.748380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60133ba-a7e2-4894-9fc4-733e77fd2093 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:50.761591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-d60133ba-a7e2-4894-9fc4-733e77fd2093 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:50.761591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:50.761591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:50.761591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:50.761591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:50.761591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:50.761591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:50.761591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:50.761591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:50.761591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:50.761591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:50.761591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:50.761591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:56:50.761591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:56:50.761591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:56:50.762639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:56:50.762639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:50.762639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:50.762639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:50.762639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:50.762639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:50.762639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:50.762639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:50.762639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:50.762639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:50.762639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:50.762639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:50.762639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:50.762639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:50.762639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:50.762639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:50.762639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:50.768443 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:56:50.768443 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:56:50.768443 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:56:50.768443 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:56:50.768443 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:56:50.768443 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:56:50.768443 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:56:50.768443 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:56:50.768443 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 556/1096] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:56:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:56:50.769060 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e066c-811b-4036-85d3-1cd7b57f3c7d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:50.854542 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:50.867156 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e066c-811b-4036-85d3-1cd7b57f3c7d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:50.899423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c143d707-9afc-4f66-89a9-6553c3d5b225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:50.900118 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143d707-9afc-4f66-89a9-6553c3d5b225 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:50.900405 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143d707-9afc-4f66-89a9-6553c3d5b225 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:50.900680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c143d707-9afc-4f66-89a9-6553c3d5b225 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:50.901457 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c143d707-9afc-4f66-89a9-6553c3d5b225 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:50.901767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c143d707-9afc-4f66-89a9-6553c3d5b225 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:50.903215 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c143d707-9afc-4f66-89a9-6553c3d5b225 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:50.920369 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 541/1097] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:56:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:50.921549 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60133ba-a7e2-4894-9fc4-733e77fd2093 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:51.026094 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60133ba-a7e2-4894-9fc4-733e77fd2093 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:51.059113 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf2ac6-f460-4ad3-abab-7e6df3cb796b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:51.059761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf2ac6-f460-4ad3-abab-7e6df3cb796b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:51.060201 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf2ac6-f460-4ad3-abab-7e6df3cb796b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:51.060487 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf2ac6-f460-4ad3-abab-7e6df3cb796b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:51.061326 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf2ac6-f460-4ad3-abab-7e6df3cb796b None admin] RBAC: Adding query filter params (role.id=039f54fd5a574af19dad80e091f25108, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=ac717fcd9fc1499db12cf852d45572a8) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:51.061643 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf2ac6-f460-4ad3-abab-7e6df3cb796b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:51.063086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf2ac6-f460-4ad3-abab-7e6df3cb796b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:51.068889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 557/1098] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:56:50 2022] GET /identity/v3/role_assignments?user.id=ac717fcd9fc1499db12cf852d45572a8&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=039f54fd5a574af19dad80e091f25108 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:51.862392 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cd156-1b3d-4165-abf8-c3327db0cc67 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:51.862897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cd156-1b3d-4165-abf8-c3327db0cc67 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:51.862999 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cd156-1b3d-4165-abf8-c3327db0cc67 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:51.864878 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 542/1099] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:51.870166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39499373-26a5-4ba3-81bb-87b2ee1b6653 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:51.870463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39499373-26a5-4ba3-81bb-87b2ee1b6653 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:51.870748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39499373-26a5-4ba3-81bb-87b2ee1b6653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:51.887279 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-39499373-26a5-4ba3-81bb-87b2ee1b6653 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:51.888780 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-39499373-26a5-4ba3-81bb-87b2ee1b6653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:51.891489 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:51.953909 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 558/1100] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:51.977572 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cd156-1b3d-4165-abf8-c3327db0cc67 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:52.078941 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cd156-1b3d-4165-abf8-c3327db0cc67 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:52.112316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7aaf1-7c76-418d-a67f-5050aa524407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:52.113150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7aaf1-7c76-418d-a67f-5050aa524407 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:52.113393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7aaf1-7c76-418d-a67f-5050aa524407 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:52.113673 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7aaf1-7c76-418d-a67f-5050aa524407 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:52.123468 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf7aaf1-7c76-418d-a67f-5050aa524407 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:56:52.131669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-0bf7aaf1-7c76-418d-a67f-5050aa524407 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:56:52.131669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:56:52.131669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:56:52.131669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:56:52.131669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:56:52.131669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:56:52.131669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:56:52.131669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:56:52.131669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:56:52.131669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:56:52.131669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:56:52.131669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:56:52.131669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:56:52.131669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:56:52.131669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:56:52.131669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:56:52.132844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:56:52.132844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:56:52.132844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:56:52.132844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:56:52.132844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:56:52.132844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:56:52.132844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:56:52.132844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:56:52.132844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:56:52.132844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:56:52.132844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:56:52.132844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:56:52.132844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:56:52.132844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:56:52.132844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:56:52.132844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:56:52.133733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:56:52.133733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:56:52.133733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:56:52.133733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:56:52.133733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:56:52.133733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:56:52.133733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:56:52.133733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:56:52.133733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:56:52.133733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:56:52.133733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:56:52.133733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:56:52.133733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 543/1101] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:56:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:56:52.138954 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39499373-26a5-4ba3-81bb-87b2ee1b6653 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:52.235548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39499373-26a5-4ba3-81bb-87b2ee1b6653 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:52.268095 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1161b55-04d4-4c17-b84a-b176d1465be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:52.268752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1161b55-04d4-4c17-b84a-b176d1465be3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:52.269040 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1161b55-04d4-4c17-b84a-b176d1465be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:52.269319 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1161b55-04d4-4c17-b84a-b176d1465be3 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:52.270062 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1161b55-04d4-4c17-b84a-b176d1465be3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:52.270376 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1161b55-04d4-4c17-b84a-b176d1465be3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:52.271862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1161b55-04d4-4c17-b84a-b176d1465be3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:52.279653 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 559/1102] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:56:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:52.285273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7aaf1-7c76-418d-a67f-5050aa524407 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:52.296950 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:52.383621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7aaf1-7c76-418d-a67f-5050aa524407 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:52.416576 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0377b0d-b61e-425f-ac88-61ba0ef9dd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:52.417432 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0377b0d-b61e-425f-ac88-61ba0ef9dd9f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:52.417714 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0377b0d-b61e-425f-ac88-61ba0ef9dd9f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:52.417989 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0377b0d-b61e-425f-ac88-61ba0ef9dd9f None admin] PATH_INFO: `/v3/users/f89df19d05e54b5c8df015ebd9e22286` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:52.419148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0377b0d-b61e-425f-ac88-61ba0ef9dd9f None admin] RBAC: Authorizing `identity:get_user(user_id=f89df19d05e54b5c8df015ebd9e22286)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:52.420608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0377b0d-b61e-425f-ac88-61ba0ef9dd9f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:52.422236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 544/1103] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:56:52 2022] GET /identity/v3/users/f89df19d05e54b5c8df015ebd9e22286 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:52.427960 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1161b55-04d4-4c17-b84a-b176d1465be3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:52.430829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:52.525355 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1161b55-04d4-4c17-b84a-b176d1465be3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:52.557689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f42bd63-f167-43fb-a6bd-b5586782dd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:52.558328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42bd63-f167-43fb-a6bd-b5586782dd97 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:52.558611 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42bd63-f167-43fb-a6bd-b5586782dd97 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:52.558886 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42bd63-f167-43fb-a6bd-b5586782dd97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:52.622795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f42bd63-f167-43fb-a6bd-b5586782dd97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:52.624342 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f42bd63-f167-43fb-a6bd-b5586782dd97 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:52.626100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:52.719363 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 560/1104] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:56:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 13:56:52.725102 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0377b0d-b61e-425f-ac88-61ba0ef9dd9f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:52.798327 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:52.820839 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0377b0d-b61e-425f-ac88-61ba0ef9dd9f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:52.852865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc4c7c-9730-4eab-b8f3-a6c61018672b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:52.853701 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc4c7c-9730-4eab-b8f3-a6c61018672b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:52.853984 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc4c7c-9730-4eab-b8f3-a6c61018672b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:52.854260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc4c7c-9730-4eab-b8f3-a6c61018672b None admin] PATH_INFO: `/v3/projects/70aab481bb27420cbd2f531a01d3e08c` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:52.855415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc4c7c-9730-4eab-b8f3-a6c61018672b None admin] RBAC: Authorizing `identity:get_project(project_id=70aab481bb27420cbd2f531a01d3e08c)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:52.856852 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc4c7c-9730-4eab-b8f3-a6c61018672b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:52.858496 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 545/1105] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:56:52 2022] GET /identity/v3/projects/70aab481bb27420cbd2f531a01d3e08c => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:52.865123 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f42bd63-f167-43fb-a6bd-b5586782dd97 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:52.960268 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f42bd63-f167-43fb-a6bd-b5586782dd97 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:52.961782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:52.995059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32aa56-52fc-469d-af73-218de1725de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:52.995647 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32aa56-52fc-469d-af73-218de1725de7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:52.995936 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32aa56-52fc-469d-af73-218de1725de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:52.996246 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32aa56-52fc-469d-af73-218de1725de7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:52.997077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32aa56-52fc-469d-af73-218de1725de7 None admin] RBAC: Adding query filter params (role.id=c60a99de086f45c6820f7872a6b43feb, scope.project.id=70aab481bb27420cbd2f531a01d3e08c, user.id=f89df19d05e54b5c8df015ebd9e22286) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:52.997392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32aa56-52fc-469d-af73-218de1725de7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:52.998828 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32aa56-52fc-469d-af73-218de1725de7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:53.004535 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 561/1106] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:56:52 2022] GET /identity/v3/role_assignments?user.id=f89df19d05e54b5c8df015ebd9e22286&scope.project.id=70aab481bb27420cbd2f531a01d3e08c&role.id=c60a99de086f45c6820f7872a6b43feb => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:55.006735 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b05dd0-5f99-408f-9c39-c5608b5ada15 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:55.007229 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b05dd0-5f99-408f-9c39-c5608b5ada15 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:55.007322 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b05dd0-5f99-408f-9c39-c5608b5ada15 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:55.008671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 546/1107] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:55.013807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10625dfb-66f0-4bdf-a573-00c8d9d5da5d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:55.014210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10625dfb-66f0-4bdf-a573-00c8d9d5da5d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:55.014363 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10625dfb-66f0-4bdf-a573-00c8d9d5da5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:55.030563 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-10625dfb-66f0-4bdf-a573-00c8d9d5da5d None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:55.031732 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-10625dfb-66f0-4bdf-a573-00c8d9d5da5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:55.096748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 562/1108] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 13:56:55.121685 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b05dd0-5f99-408f-9c39-c5608b5ada15 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:55.218761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b05dd0-5f99-408f-9c39-c5608b5ada15 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:55.251243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e598a-0bbb-4900-8bae-5187d596dca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:55.252036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e598a-0bbb-4900-8bae-5187d596dca0 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:55.252319 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e598a-0bbb-4900-8bae-5187d596dca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:55.252599 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e598a-0bbb-4900-8bae-5187d596dca0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:55.253740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e598a-0bbb-4900-8bae-5187d596dca0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:55.255194 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e598a-0bbb-4900-8bae-5187d596dca0 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:55.256789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 547/1109] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:56:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:55.262398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10625dfb-66f0-4bdf-a573-00c8d9d5da5d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:55.350816 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:55.356797 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10625dfb-66f0-4bdf-a573-00c8d9d5da5d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:55.391021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51184f39-3f28-474a-933c-f9242e35fd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:55.391680 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51184f39-3f28-474a-933c-f9242e35fd1a None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:55.391983 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51184f39-3f28-474a-933c-f9242e35fd1a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:55.392263 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51184f39-3f28-474a-933c-f9242e35fd1a None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:55.392963 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51184f39-3f28-474a-933c-f9242e35fd1a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:55.394383 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51184f39-3f28-474a-933c-f9242e35fd1a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:55.416388 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 563/1110] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Jul 18 13:56:55 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:56:57.464569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361f261-bee7-4223-aacd-43f6e777b1bb None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:57.465121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361f261-bee7-4223-aacd-43f6e777b1bb None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:57.465121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361f261-bee7-4223-aacd-43f6e777b1bb None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:57.466464 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 548/1111] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:56:57.471721 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e45198-e499-4d9f-abc0-d489b81c6d58 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:57.472162 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e45198-e499-4d9f-abc0-d489b81c6d58 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:57.472326 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e45198-e499-4d9f-abc0-d489b81c6d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:57.490115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-83e45198-e499-4d9f-abc0-d489b81c6d58 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:56:57.491342 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-83e45198-e499-4d9f-abc0-d489b81c6d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:56:57.559180 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 564/1112] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:56:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:56:57.582190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361f261-bee7-4223-aacd-43f6e777b1bb None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:57.596674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:57.683993 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361f261-bee7-4223-aacd-43f6e777b1bb None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:57.716651 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937c572-901b-4f43-bb19-28fdb7581568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:57.717436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937c572-901b-4f43-bb19-28fdb7581568 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:57.717723 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937c572-901b-4f43-bb19-28fdb7581568 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:57.718001 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937c572-901b-4f43-bb19-28fdb7581568 None admin] PATH_INFO: `/v3/roles/580aae575f414b3c9b4ccf1e27e0b640` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:57.719469 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937c572-901b-4f43-bb19-28fdb7581568 None admin] RBAC: Authorizing `identity:get_role(role_id=580aae575f414b3c9b4ccf1e27e0b640)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:57.720866 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937c572-901b-4f43-bb19-28fdb7581568 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:57.722226 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 549/1113] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:56:57 2022] GET /identity/v3/roles/580aae575f414b3c9b4ccf1e27e0b640 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:57.728841 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e45198-e499-4d9f-abc0-d489b81c6d58 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:57.807836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:57.839630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e45198-e499-4d9f-abc0-d489b81c6d58 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:57.875405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694db51c-f5e3-4243-b7ed-839ae1710e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:57.876310 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694db51c-f5e3-4243-b7ed-839ae1710e3c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:57.876601 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694db51c-f5e3-4243-b7ed-839ae1710e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:57.876884 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694db51c-f5e3-4243-b7ed-839ae1710e3c None admin] PATH_INFO: `/v3/users/cfb9a3c32455463c8f18a07511259f9c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:57.889941 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694db51c-f5e3-4243-b7ed-839ae1710e3c None admin] RBAC: Authorizing `identity:get_user(user_id=cfb9a3c32455463c8f18a07511259f9c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:57.891414 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694db51c-f5e3-4243-b7ed-839ae1710e3c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:57.893116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 565/1114] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:56:57 2022] GET /identity/v3/users/cfb9a3c32455463c8f18a07511259f9c => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:57.899412 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937c572-901b-4f43-bb19-28fdb7581568 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:57.981941 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:58.001602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937c572-901b-4f43-bb19-28fdb7581568 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:58.035503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3e1b9-beec-45d5-b1a2-53f6eb4c39ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:58.036735 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3e1b9-beec-45d5-b1a2-53f6eb4c39ee None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:58.037026 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3e1b9-beec-45d5-b1a2-53f6eb4c39ee None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:58.037309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3e1b9-beec-45d5-b1a2-53f6eb4c39ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:58.103518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a3e1b9-beec-45d5-b1a2-53f6eb4c39ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:58.105006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a3e1b9-beec-45d5-b1a2-53f6eb4c39ee None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:58.181206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:56:58.202374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 550/1115] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:56:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 13:56:58.209152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694db51c-f5e3-4243-b7ed-839ae1710e3c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:58.306026 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694db51c-f5e3-4243-b7ed-839ae1710e3c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:58.339655 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39060663-e0c6-48a2-b543-8e361f6ebcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:58.340543 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39060663-e0c6-48a2-b543-8e361f6ebcab None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:58.340832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39060663-e0c6-48a2-b543-8e361f6ebcab None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:58.341110 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39060663-e0c6-48a2-b543-8e361f6ebcab None admin] PATH_INFO: `/v3/projects/70aab481bb27420cbd2f531a01d3e08c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:58.342277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39060663-e0c6-48a2-b543-8e361f6ebcab None admin] RBAC: Authorizing `identity:get_project(project_id=70aab481bb27420cbd2f531a01d3e08c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:58.343709 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39060663-e0c6-48a2-b543-8e361f6ebcab None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:58.345446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 566/1116] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:56:58 2022] GET /identity/v3/projects/70aab481bb27420cbd2f531a01d3e08c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:56:58.351646 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3e1b9-beec-45d5-b1a2-53f6eb4c39ee None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:56:58.455751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3e1b9-beec-45d5-b1a2-53f6eb4c39ee None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:56:58.490722 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad1edf-acc8-451f-8ec7-63d96f3adb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:56:58.491378 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad1edf-acc8-451f-8ec7-63d96f3adb6a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:56:58.491656 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad1edf-acc8-451f-8ec7-63d96f3adb6a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:56:58.491931 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad1edf-acc8-451f-8ec7-63d96f3adb6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:56:58.492803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad1edf-acc8-451f-8ec7-63d96f3adb6a None admin] RBAC: Adding query filter params (role.id=580aae575f414b3c9b4ccf1e27e0b640, scope.project.id=70aab481bb27420cbd2f531a01d3e08c, user.id=cfb9a3c32455463c8f18a07511259f9c) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:56:58.493117 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad1edf-acc8-451f-8ec7-63d96f3adb6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:56:58.494578 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad1edf-acc8-451f-8ec7-63d96f3adb6a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:56:58.500819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 551/1117] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:56:58 2022] GET /identity/v3/role_assignments?user.id=cfb9a3c32455463c8f18a07511259f9c&scope.project.id=70aab481bb27420cbd2f531a01d3e08c&role.id=580aae575f414b3c9b4ccf1e27e0b640 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:00.472154 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ef03b-15c7-4345-9d61-05ebcaf9d4a5 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:00.472778 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ef03b-15c7-4345-9d61-05ebcaf9d4a5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:00.472778 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ef03b-15c7-4345-9d61-05ebcaf9d4a5 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:00.474620 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 567/1118] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:57:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:57:00.479708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751e89b-d676-4355-b69e-679934ada1f7 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:00.480133 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751e89b-d676-4355-b69e-679934ada1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:00.480308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751e89b-d676-4355-b69e-679934ada1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:00.497106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-a751e89b-d676-4355-b69e-679934ada1f7 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:57:00.498285 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a751e89b-d676-4355-b69e-679934ada1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:57:00.537475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:00.563635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 552/1119] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:57:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:57:00.586781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ef03b-15c7-4345-9d61-05ebcaf9d4a5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:00.597457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:00.687206 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ef03b-15c7-4345-9d61-05ebcaf9d4a5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:00.720712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce2854-bf0d-4df6-b1ec-115251deca7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:00.721554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce2854-bf0d-4df6-b1ec-115251deca7b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:00.721844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce2854-bf0d-4df6-b1ec-115251deca7b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:00.722126 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce2854-bf0d-4df6-b1ec-115251deca7b None admin] PATH_INFO: `/v3/roles/580aae575f414b3c9b4ccf1e27e0b640` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:00.724403 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce2854-bf0d-4df6-b1ec-115251deca7b None admin] RBAC: Authorizing `identity:get_role(role_id=580aae575f414b3c9b4ccf1e27e0b640)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:00.725539 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce2854-bf0d-4df6-b1ec-115251deca7b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:00.726899 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 568/1120] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:57:00 2022] GET /identity/v3/roles/580aae575f414b3c9b4ccf1e27e0b640 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:00.732649 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751e89b-d676-4355-b69e-679934ada1f7 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:00.826162 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751e89b-d676-4355-b69e-679934ada1f7 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:00.858733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b005378-54d6-4876-b66a-6c8e194db3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:00.859582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b005378-54d6-4876-b66a-6c8e194db3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:00.859868 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b005378-54d6-4876-b66a-6c8e194db3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:00.860173 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b005378-54d6-4876-b66a-6c8e194db3e6 None admin] PATH_INFO: `/v3/users/cfb9a3c32455463c8f18a07511259f9c` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:00.861342 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b005378-54d6-4876-b66a-6c8e194db3e6 None admin] RBAC: Authorizing `identity:get_user(user_id=cfb9a3c32455463c8f18a07511259f9c)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:00.862759 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b005378-54d6-4876-b66a-6c8e194db3e6 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:00.864911 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 553/1121] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:57:00 2022] GET /identity/v3/users/cfb9a3c32455463c8f18a07511259f9c => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:00.870372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce2854-bf0d-4df6-b1ec-115251deca7b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:00.944214 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:00.966907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce2854-bf0d-4df6-b1ec-115251deca7b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:01.000270 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc6fee-0e0b-4c33-8d2c-fe17639b7a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:01.001075 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc6fee-0e0b-4c33-8d2c-fe17639b7a17 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:01.001362 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc6fee-0e0b-4c33-8d2c-fe17639b7a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:01.001641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc6fee-0e0b-4c33-8d2c-fe17639b7a17 None admin] PATH_INFO: `/v3/projects/70aab481bb27420cbd2f531a01d3e08c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:01.002826 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc6fee-0e0b-4c33-8d2c-fe17639b7a17 None admin] RBAC: Authorizing `identity:get_project(project_id=70aab481bb27420cbd2f531a01d3e08c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:01.004292 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc6fee-0e0b-4c33-8d2c-fe17639b7a17 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:01.006640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 569/1122] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:57:00 2022] GET /identity/v3/projects/70aab481bb27420cbd2f531a01d3e08c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:01.012480 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b005378-54d6-4876-b66a-6c8e194db3e6 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:01.054596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:01.108269 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b005378-54d6-4876-b66a-6c8e194db3e6 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:01.141194 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02fd03-b2c6-49c7-b097-cac8ac32fdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:01.142066 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02fd03-b2c6-49c7-b097-cac8ac32fdde None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:01.142351 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02fd03-b2c6-49c7-b097-cac8ac32fdde None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:01.142623 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02fd03-b2c6-49c7-b097-cac8ac32fdde None admin] PATH_INFO: `/v3/projects/70aab481bb27420cbd2f531a01d3e08c/users/cfb9a3c32455463c8f18a07511259f9c/roles/580aae575f414b3c9b4ccf1e27e0b640` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:01.144814 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02fd03-b2c6-49c7-b097-cac8ac32fdde None admin] RBAC: Authorizing `identity:create_grant(project_id=70aab481bb27420cbd2f531a01d3e08c, user_id=cfb9a3c32455463c8f18a07511259f9c, role_id=580aae575f414b3c9b4ccf1e27e0b640)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:01.146268 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02fd03-b2c6-49c7-b097-cac8ac32fdde None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:01.158434 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 554/1123] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:57:01 2022] PUT /identity/v3/projects/70aab481bb27420cbd2f531a01d3e08c/users/cfb9a3c32455463c8f18a07511259f9c/roles/580aae575f414b3c9b4ccf1e27e0b640 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:57:03.125297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491515f-9be8-4ce7-b52f-43e7ede747ff None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:03.125856 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491515f-9be8-4ce7-b52f-43e7ede747ff None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:03.125856 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491515f-9be8-4ce7-b52f-43e7ede747ff None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:03.127238 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 570/1124] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:57:03.132463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8799934-0898-4f4e-b2ed-a486ed1c76c1 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:03.132875 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8799934-0898-4f4e-b2ed-a486ed1c76c1 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:03.133028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8799934-0898-4f4e-b2ed-a486ed1c76c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:03.157686 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b8799934-0898-4f4e-b2ed-a486ed1c76c1 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:57:03.159728 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b8799934-0898-4f4e-b2ed-a486ed1c76c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:57:03.230039 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 555/1125] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:57:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:57:03.253079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491515f-9be8-4ce7-b52f-43e7ede747ff None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:03.351626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491515f-9be8-4ce7-b52f-43e7ede747ff None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:03.383754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee331a-25d9-4041-9569-247ffff3744d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:03.384625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee331a-25d9-4041-9569-247ffff3744d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:03.384913 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee331a-25d9-4041-9569-247ffff3744d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:03.385189 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee331a-25d9-4041-9569-247ffff3744d None admin] PATH_INFO: `/v3/roles/580aae575f414b3c9b4ccf1e27e0b640` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:03.386645 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee331a-25d9-4041-9569-247ffff3744d None admin] RBAC: Authorizing `identity:get_role(role_id=580aae575f414b3c9b4ccf1e27e0b640)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:03.388031 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee331a-25d9-4041-9569-247ffff3744d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:03.389367 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 571/1126] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:57:03 2022] GET /identity/v3/roles/580aae575f414b3c9b4ccf1e27e0b640 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:03.395262 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8799934-0898-4f4e-b2ed-a486ed1c76c1 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:03.425480 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:03.493907 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8799934-0898-4f4e-b2ed-a486ed1c76c1 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:03.529111 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ebd5a-1d22-4b49-b0a9-8c2d8952b4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:03.529996 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ebd5a-1d22-4b49-b0a9-8c2d8952b4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:03.530284 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ebd5a-1d22-4b49-b0a9-8c2d8952b4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:03.530561 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ebd5a-1d22-4b49-b0a9-8c2d8952b4b3 None admin] PATH_INFO: `/v3/users/cfb9a3c32455463c8f18a07511259f9c` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:03.531762 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ebd5a-1d22-4b49-b0a9-8c2d8952b4b3 None admin] RBAC: Authorizing `identity:get_user(user_id=cfb9a3c32455463c8f18a07511259f9c)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:03.533249 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ebd5a-1d22-4b49-b0a9-8c2d8952b4b3 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:03.534893 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 556/1127] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:57:03 2022] GET /identity/v3/users/cfb9a3c32455463c8f18a07511259f9c => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:03.541348 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee331a-25d9-4041-9569-247ffff3744d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:03.557248 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:03.638460 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee331a-25d9-4041-9569-247ffff3744d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:03.671262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c5cf6-dcee-44f8-a3de-7bdba3a4929f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:03.671896 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c5cf6-dcee-44f8-a3de-7bdba3a4929f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:03.672209 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c5cf6-dcee-44f8-a3de-7bdba3a4929f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:03.672482 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c5cf6-dcee-44f8-a3de-7bdba3a4929f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:03.735935 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2c5cf6-dcee-44f8-a3de-7bdba3a4929f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:03.737393 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2c5cf6-dcee-44f8-a3de-7bdba3a4929f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:03.750160 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:03.830688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 572/1128] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:57:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 13:57:03.837313 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ebd5a-1d22-4b49-b0a9-8c2d8952b4b3 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:03.930105 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:03.932960 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ebd5a-1d22-4b49-b0a9-8c2d8952b4b3 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:03.966804 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08e41c-3284-4e42-96b7-6f0b63985e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:03.967662 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08e41c-3284-4e42-96b7-6f0b63985e7b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:03.967977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08e41c-3284-4e42-96b7-6f0b63985e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:03.968264 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08e41c-3284-4e42-96b7-6f0b63985e7b None admin] PATH_INFO: `/v3/projects/70aab481bb27420cbd2f531a01d3e08c` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:03.969437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08e41c-3284-4e42-96b7-6f0b63985e7b None admin] RBAC: Authorizing `identity:get_project(project_id=70aab481bb27420cbd2f531a01d3e08c)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:03.970862 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08e41c-3284-4e42-96b7-6f0b63985e7b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:03.972580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 557/1129] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:57:03 2022] GET /identity/v3/projects/70aab481bb27420cbd2f531a01d3e08c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:03.979258 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c5cf6-dcee-44f8-a3de-7bdba3a4929f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:04.075233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c5cf6-dcee-44f8-a3de-7bdba3a4929f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:04.088668 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:04.109288 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2387f7-cfdf-40e2-882c-8a9f92a64907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:04.109927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2387f7-cfdf-40e2-882c-8a9f92a64907 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:04.110216 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2387f7-cfdf-40e2-882c-8a9f92a64907 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:04.110495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2387f7-cfdf-40e2-882c-8a9f92a64907 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:04.111321 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2387f7-cfdf-40e2-882c-8a9f92a64907 None admin] RBAC: Adding query filter params (role.id=580aae575f414b3c9b4ccf1e27e0b640, scope.project.id=70aab481bb27420cbd2f531a01d3e08c, user.id=cfb9a3c32455463c8f18a07511259f9c) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:57:04.111637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2387f7-cfdf-40e2-882c-8a9f92a64907 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:04.113100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2387f7-cfdf-40e2-882c-8a9f92a64907 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:04.118801 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 573/1130] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:57:03 2022] GET /identity/v3/role_assignments?user.id=cfb9a3c32455463c8f18a07511259f9c&scope.project.id=70aab481bb27420cbd2f531a01d3e08c&role.id=580aae575f414b3c9b4ccf1e27e0b640 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:06.123821 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a19c6-6661-4a30-9071-c654e57733f6 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:06.124380 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a19c6-6661-4a30-9071-c654e57733f6 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:06.124489 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a19c6-6661-4a30-9071-c654e57733f6 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:06.125778 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 558/1131] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:57:06.131039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012aa4b-0680-4099-804f-8583cd1540a8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:06.131458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012aa4b-0680-4099-804f-8583cd1540a8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:06.131611 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012aa4b-0680-4099-804f-8583cd1540a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:06.148426 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6012aa4b-0680-4099-804f-8583cd1540a8 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:57:06.149604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6012aa4b-0680-4099-804f-8583cd1540a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:57:06.213609 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 574/1132] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:57:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 13:57:06.236722 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a19c6-6661-4a30-9071-c654e57733f6 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:06.335513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a19c6-6661-4a30-9071-c654e57733f6 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:06.367421 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae08e5e-c681-4127-ad75-a579f4741512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:06.368227 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae08e5e-c681-4127-ad75-a579f4741512 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:06.368517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae08e5e-c681-4127-ad75-a579f4741512 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:06.368798 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae08e5e-c681-4127-ad75-a579f4741512 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:06.369972 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae08e5e-c681-4127-ad75-a579f4741512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:06.371408 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae08e5e-c681-4127-ad75-a579f4741512 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:06.373015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 559/1133] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:57:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:06.378628 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012aa4b-0680-4099-804f-8583cd1540a8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:06.474247 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012aa4b-0680-4099-804f-8583cd1540a8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:06.481849 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:06.508398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce9b8aa-e807-46b2-a8ef-72fa8f8037e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:06.509047 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9b8aa-e807-46b2-a8ef-72fa8f8037e6 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:06.509336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9b8aa-e807-46b2-a8ef-72fa8f8037e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:06.509616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9b8aa-e807-46b2-a8ef-72fa8f8037e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:06.510323 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce9b8aa-e807-46b2-a8ef-72fa8f8037e6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:06.511747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce9b8aa-e807-46b2-a8ef-72fa8f8037e6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:06.536654 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 575/1134] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Jul 18 13:57:06 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:57:08.539737 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c90177-577b-4732-b881-4cba69493aad None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:08.540344 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c90177-577b-4732-b881-4cba69493aad None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:08.540344 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c90177-577b-4732-b881-4cba69493aad None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:08.541708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 560/1135] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:57:08.546746 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afd741-27b1-4ea1-a657-5e99125e3393 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:08.547178 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afd741-27b1-4ea1-a657-5e99125e3393 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:08.547317 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afd741-27b1-4ea1-a657-5e99125e3393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:08.564336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c2afd741-27b1-4ea1-a657-5e99125e3393 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:57:08.565518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c2afd741-27b1-4ea1-a657-5e99125e3393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:57:08.631559 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 576/1136] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:57:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:57:08.655410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c90177-577b-4732-b881-4cba69493aad None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:08.691317 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:08.754899 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c90177-577b-4732-b881-4cba69493aad None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:08.787327 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10f1bc-c4c5-45b9-b2c9-f12f612d4ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:08.788135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10f1bc-c4c5-45b9-b2c9-f12f612d4ece None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:08.788426 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10f1bc-c4c5-45b9-b2c9-f12f612d4ece None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:08.788707 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10f1bc-c4c5-45b9-b2c9-f12f612d4ece None admin] PATH_INFO: `/v3/domains/default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:08.789915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10f1bc-c4c5-45b9-b2c9-f12f612d4ece None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:08.791289 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10f1bc-c4c5-45b9-b2c9-f12f612d4ece None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:08.792906 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 561/1137] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:57:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:08.798693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afd741-27b1-4ea1-a657-5e99125e3393 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:08.882831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:08.895878 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afd741-27b1-4ea1-a657-5e99125e3393 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:08.928394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e85122-35c9-4ee8-ad7b-911c90db0726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:08.929047 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e85122-35c9-4ee8-ad7b-911c90db0726 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:08.929331 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e85122-35c9-4ee8-ad7b-911c90db0726 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:08.929608 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e85122-35c9-4ee8-ad7b-911c90db0726 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:08.930316 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e85122-35c9-4ee8-ad7b-911c90db0726 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:08.931742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e85122-35c9-4ee8-ad7b-911c90db0726 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:08.951060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 577/1138] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Jul 18 13:57:08 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:57:10.971591 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860d4b3-c963-4c07-9abb-39ab7e7fd5ae None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:10.972052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860d4b3-c963-4c07-9abb-39ab7e7fd5ae None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:10.972183 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860d4b3-c963-4c07-9abb-39ab7e7fd5ae None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:10.974091 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 562/1139] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:57:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:57:10.979370 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690e6f4-d2aa-4cda-9382-63b74e28dd9e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:10.979790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690e6f4-d2aa-4cda-9382-63b74e28dd9e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:10.979935 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690e6f4-d2aa-4cda-9382-63b74e28dd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:10.996864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c690e6f4-d2aa-4cda-9382-63b74e28dd9e None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:57:10.998042 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c690e6f4-d2aa-4cda-9382-63b74e28dd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:57:11.061628 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 578/1140] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:57:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:57:11.084560 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860d4b3-c963-4c07-9abb-39ab7e7fd5ae None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:11.183703 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860d4b3-c963-4c07-9abb-39ab7e7fd5ae None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:11.186013 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:11.222824 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e073115-de61-42a1-a465-51f74531f713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:11.223540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e073115-de61-42a1-a465-51f74531f713 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:11.223832 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e073115-de61-42a1-a465-51f74531f713 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:11.224149 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e073115-de61-42a1-a465-51f74531f713 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:11.234130 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e073115-de61-42a1-a465-51f74531f713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:57:11.243403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-8e073115-de61-42a1-a465-51f74531f713 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:57:11.243403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:57:11.243403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:57:11.243403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:57:11.243403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:57:11.243403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:57:11.243403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:57:11.243403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:57:11.243403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:57:11.243403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:57:11.243403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:57:11.243403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:57:11.243403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:57:11.243403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:57:11.243403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:57:11.243403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:57:11.244653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:57:11.244653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:57:11.244653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:57:11.244653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:57:11.244653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:57:11.244653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:57:11.244653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:57:11.244653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:57:11.244653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:57:11.244653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:57:11.244653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:57:11.244653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:57:11.244653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:57:11.244653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:57:11.244653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:57:11.244653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:57:11.245776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:57:11.245776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:57:11.245776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:57:11.245776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:57:11.245776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:57:11.245776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:57:11.245776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:57:11.245776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:57:11.245776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:57:11.245776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:57:11.245776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:57:11.245776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:57:11.245776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 563/1141] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:57:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:57:11.253027 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690e6f4-d2aa-4cda-9382-63b74e28dd9e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:11.326930 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:11.355264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690e6f4-d2aa-4cda-9382-63b74e28dd9e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:11.400524 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ee6a9-12a5-4f55-a7d7-3e123492caa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:11.401180 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ee6a9-12a5-4f55-a7d7-3e123492caa6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:11.401540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ee6a9-12a5-4f55-a7d7-3e123492caa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:11.402052 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ee6a9-12a5-4f55-a7d7-3e123492caa6 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:11.403747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ee6a9-12a5-4f55-a7d7-3e123492caa6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:57:11.404508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ee6a9-12a5-4f55-a7d7-3e123492caa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:11.405886 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ee6a9-12a5-4f55-a7d7-3e123492caa6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:11.414029 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 579/1142] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:57:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:11.420026 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e073115-de61-42a1-a465-51f74531f713 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:11.518230 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e073115-de61-42a1-a465-51f74531f713 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:11.551552 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16fa5d3-629f-4dc8-965a-f38e0cff116e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:11.552434 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fa5d3-629f-4dc8-965a-f38e0cff116e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:11.552771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fa5d3-629f-4dc8-965a-f38e0cff116e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:11.553052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fa5d3-629f-4dc8-965a-f38e0cff116e None admin] PATH_INFO: `/v3/users/0cac3d3a76a34aa790f5f4d6959d8a92` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:11.565903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16fa5d3-629f-4dc8-965a-f38e0cff116e None admin] RBAC: Authorizing `identity:get_user(user_id=0cac3d3a76a34aa790f5f4d6959d8a92)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:11.567371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16fa5d3-629f-4dc8-965a-f38e0cff116e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:11.569073 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 564/1143] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:57:11 2022] GET /identity/v3/users/0cac3d3a76a34aa790f5f4d6959d8a92 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:11.575133 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ee6a9-12a5-4f55-a7d7-3e123492caa6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:11.677270 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ee6a9-12a5-4f55-a7d7-3e123492caa6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:11.709317 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782fde5-7161-43fb-a995-41483d934781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:11.709964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782fde5-7161-43fb-a995-41483d934781 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:11.710254 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782fde5-7161-43fb-a995-41483d934781 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:11.710538 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782fde5-7161-43fb-a995-41483d934781 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:11.713901 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:11.776022 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7782fde5-7161-43fb-a995-41483d934781 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:11.777415 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7782fde5-7161-43fb-a995-41483d934781 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:11.872066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 580/1144] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:57:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 13:57:11.877991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16fa5d3-629f-4dc8-965a-f38e0cff116e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:11.908528 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:11.984396 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16fa5d3-629f-4dc8-965a-f38e0cff116e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:12.017381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399c053-6ced-4b2f-ad28-6563bb220fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:12.018230 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399c053-6ced-4b2f-ad28-6563bb220fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:12.018515 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399c053-6ced-4b2f-ad28-6563bb220fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:12.018792 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399c053-6ced-4b2f-ad28-6563bb220fe7 None admin] PATH_INFO: `/v3/projects/5c6c6221968c40d38eb3255fb497ecc3` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:12.019968 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399c053-6ced-4b2f-ad28-6563bb220fe7 None admin] RBAC: Authorizing `identity:get_project(project_id=5c6c6221968c40d38eb3255fb497ecc3)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:12.021401 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399c053-6ced-4b2f-ad28-6563bb220fe7 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:12.023081 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 565/1145] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:57:11 2022] GET /identity/v3/projects/5c6c6221968c40d38eb3255fb497ecc3 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:12.029183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782fde5-7161-43fb-a995-41483d934781 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:12.061961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:12.123581 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782fde5-7161-43fb-a995-41483d934781 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:12.155821 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55deca-e584-4c6b-be73-897ca377cc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:12.156493 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55deca-e584-4c6b-be73-897ca377cc7c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:12.156772 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55deca-e584-4c6b-be73-897ca377cc7c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:12.157054 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55deca-e584-4c6b-be73-897ca377cc7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:12.157880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55deca-e584-4c6b-be73-897ca377cc7c None admin] RBAC: Adding query filter params (role.id=c60a99de086f45c6820f7872a6b43feb, scope.project.id=5c6c6221968c40d38eb3255fb497ecc3, user.id=0cac3d3a76a34aa790f5f4d6959d8a92) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:57:12.158195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55deca-e584-4c6b-be73-897ca377cc7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:12.159622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55deca-e584-4c6b-be73-897ca377cc7c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:12.164892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 581/1146] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:57:12 2022] GET /identity/v3/role_assignments?user.id=0cac3d3a76a34aa790f5f4d6959d8a92&scope.project.id=5c6c6221968c40d38eb3255fb497ecc3&role.id=c60a99de086f45c6820f7872a6b43feb => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:14.094220 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139d159-0823-41d9-9be5-2dad7d0a12b9 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:14.094725 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139d159-0823-41d9-9be5-2dad7d0a12b9 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:14.094810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139d159-0823-41d9-9be5-2dad7d0a12b9 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:14.096179 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 566/1147] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:57:14.101776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177eda37-6647-4593-85c3-04c877c3ed65 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:14.102203 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177eda37-6647-4593-85c3-04c877c3ed65 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:14.102333 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177eda37-6647-4593-85c3-04c877c3ed65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:14.118956 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-177eda37-6647-4593-85c3-04c877c3ed65 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:57:14.120152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-177eda37-6647-4593-85c3-04c877c3ed65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:57:14.183579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 582/1148] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:57:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:57:14.206094 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139d159-0823-41d9-9be5-2dad7d0a12b9 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:14.292922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:14.303814 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139d159-0823-41d9-9be5-2dad7d0a12b9 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:14.336466 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e7bcb-e5fe-44b8-9803-28125eee6b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:14.337249 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e7bcb-e5fe-44b8-9803-28125eee6b3f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:14.337599 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e7bcb-e5fe-44b8-9803-28125eee6b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:14.337820 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e7bcb-e5fe-44b8-9803-28125eee6b3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:14.347544 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e7bcb-e5fe-44b8-9803-28125eee6b3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:57:14.355383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-d64e7bcb-e5fe-44b8-9803-28125eee6b3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:57:14.355383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:57:14.355383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:57:14.355383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:57:14.355383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:57:14.355383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:57:14.355383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:57:14.355383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:57:14.355383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:57:14.355383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:57:14.355383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:57:14.355383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:57:14.355383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:57:14.355383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:57:14.355383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:57:14.355383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:57:14.356581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:57:14.356581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:57:14.356581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:57:14.356581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:57:14.356581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:57:14.356581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:57:14.356581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:57:14.356581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:57:14.356581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:57:14.356581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:57:14.356581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:57:14.356581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:57:14.356581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:57:14.356581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:57:14.356581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:57:14.356581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:57:14.357779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:57:14.357779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:57:14.357779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:57:14.357779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:57:14.357779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:57:14.357779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:57:14.357779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:57:14.357779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:57:14.357779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:57:14.357779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:57:14.357779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:57:14.357779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:57:14.357779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 567/1149] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:57:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:57:14.362384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177eda37-6647-4593-85c3-04c877c3ed65 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:14.391537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:14.458536 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177eda37-6647-4593-85c3-04c877c3ed65 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:14.490685 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd74ee-036f-47a0-a23a-e6a28d74e21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:14.491337 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd74ee-036f-47a0-a23a-e6a28d74e21d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:14.491623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd74ee-036f-47a0-a23a-e6a28d74e21d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:14.491903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd74ee-036f-47a0-a23a-e6a28d74e21d None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:14.492677 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd74ee-036f-47a0-a23a-e6a28d74e21d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:57:14.492988 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd74ee-036f-47a0-a23a-e6a28d74e21d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:14.494348 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd74ee-036f-47a0-a23a-e6a28d74e21d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:14.502091 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 583/1150] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:57:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:14.507721 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e7bcb-e5fe-44b8-9803-28125eee6b3f None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:14.604767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e7bcb-e5fe-44b8-9803-28125eee6b3f None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:14.636716 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045e1cd-9cb5-459b-bc55-ee93adc18d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:14.637563 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045e1cd-9cb5-459b-bc55-ee93adc18d9c None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:14.637847 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045e1cd-9cb5-459b-bc55-ee93adc18d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:14.638129 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045e1cd-9cb5-459b-bc55-ee93adc18d9c None admin] PATH_INFO: `/v3/users/0cac3d3a76a34aa790f5f4d6959d8a92` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:14.639303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045e1cd-9cb5-459b-bc55-ee93adc18d9c None admin] RBAC: Authorizing `identity:get_user(user_id=0cac3d3a76a34aa790f5f4d6959d8a92)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:14.640747 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045e1cd-9cb5-459b-bc55-ee93adc18d9c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:14.642335 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 568/1151] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:57:14 2022] GET /identity/v3/users/0cac3d3a76a34aa790f5f4d6959d8a92 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:14.648375 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd74ee-036f-47a0-a23a-e6a28d74e21d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:14.737303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:14.746491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd74ee-036f-47a0-a23a-e6a28d74e21d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:14.779403 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf68308e-f3af-4e14-a203-9ad34d609628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:14.780282 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68308e-f3af-4e14-a203-9ad34d609628 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:14.780572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68308e-f3af-4e14-a203-9ad34d609628 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:14.780854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf68308e-f3af-4e14-a203-9ad34d609628 None admin] PATH_INFO: `/v3/projects/5c6c6221968c40d38eb3255fb497ecc3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:14.782033 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf68308e-f3af-4e14-a203-9ad34d609628 None admin] RBAC: Authorizing `identity:get_project(project_id=5c6c6221968c40d38eb3255fb497ecc3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:14.783470 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf68308e-f3af-4e14-a203-9ad34d609628 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:14.785208 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 584/1152] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:57:14 2022] GET /identity/v3/projects/5c6c6221968c40d38eb3255fb497ecc3 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:14.791059 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045e1cd-9cb5-459b-bc55-ee93adc18d9c None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:14.809545 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:14.887203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045e1cd-9cb5-459b-bc55-ee93adc18d9c None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:14.919848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e200ff-6131-4ef1-9b08-724f445e1cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:14.920758 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e200ff-6131-4ef1-9b08-724f445e1cf7 None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:14.921042 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e200ff-6131-4ef1-9b08-724f445e1cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:14.921319 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e200ff-6131-4ef1-9b08-724f445e1cf7 None admin] PATH_INFO: `/v3/projects/5c6c6221968c40d38eb3255fb497ecc3/users/0cac3d3a76a34aa790f5f4d6959d8a92/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:14.923240 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e200ff-6131-4ef1-9b08-724f445e1cf7 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c6c6221968c40d38eb3255fb497ecc3, user_id=0cac3d3a76a34aa790f5f4d6959d8a92, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:14.924698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e200ff-6131-4ef1-9b08-724f445e1cf7 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:14.937665 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 569/1153] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:57:14 2022] PUT /identity/v3/projects/5c6c6221968c40d38eb3255fb497ecc3/users/0cac3d3a76a34aa790f5f4d6959d8a92/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:57:16.926744 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e27e20-16a3-45d1-9e77-d8bf10862ec1 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:16.927309 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e27e20-16a3-45d1-9e77-d8bf10862ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:16.927309 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e27e20-16a3-45d1-9e77-d8bf10862ec1 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:16.928725 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 585/1154] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:57:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:57:16.933799 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef823136-4a56-4012-8230-fdc67b4dc5bb None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:16.934209 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef823136-4a56-4012-8230-fdc67b4dc5bb None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:16.934374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef823136-4a56-4012-8230-fdc67b4dc5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:16.951399 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ef823136-4a56-4012-8230-fdc67b4dc5bb None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:57:16.952876 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ef823136-4a56-4012-8230-fdc67b4dc5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:57:17.017129 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 570/1155] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:57:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:57:17.040540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e27e20-16a3-45d1-9e77-d8bf10862ec1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:17.138936 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e27e20-16a3-45d1-9e77-d8bf10862ec1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:17.171627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09953985-ac7c-45a7-9cd5-6a56619a5b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:17.172441 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09953985-ac7c-45a7-9cd5-6a56619a5b79 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:17.172731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09953985-ac7c-45a7-9cd5-6a56619a5b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:17.173015 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09953985-ac7c-45a7-9cd5-6a56619a5b79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:17.184604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09953985-ac7c-45a7-9cd5-6a56619a5b79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:57:17.192708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-09953985-ac7c-45a7-9cd5-6a56619a5b79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:57:17.192708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:57:17.192708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:57:17.192708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:57:17.192708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:57:17.192708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:57:17.192708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:57:17.192708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:57:17.192708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:57:17.192708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:57:17.192708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:57:17.192708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:57:17.192708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:57:17.192708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:57:17.192708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:57:17.192708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:57:17.193817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:57:17.193817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:57:17.193817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:57:17.193817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:57:17.193817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:57:17.193817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:57:17.193817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:57:17.193817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:57:17.193817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:57:17.193817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:57:17.193817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:57:17.193817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:57:17.193817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:57:17.193817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:57:17.193817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:57:17.193817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:57:17.194877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:57:17.194877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:57:17.194877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:57:17.194877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:57:17.194877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:57:17.194877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:57:17.194877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:57:17.194877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:57:17.194877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:57:17.194877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:57:17.194877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:57:17.194877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:57:17.194877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 586/1156] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:57:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:57:17.199662 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef823136-4a56-4012-8230-fdc67b4dc5bb None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:17.210139 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:17.298208 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef823136-4a56-4012-8230-fdc67b4dc5bb None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:17.332159 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17bc80-850e-401d-a429-e82d4e2b093b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:17.332822 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17bc80-850e-401d-a429-e82d4e2b093b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:17.333115 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17bc80-850e-401d-a429-e82d4e2b093b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:17.333398 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17bc80-850e-401d-a429-e82d4e2b093b None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:17.334139 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b17bc80-850e-401d-a429-e82d4e2b093b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:57:17.334449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b17bc80-850e-401d-a429-e82d4e2b093b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:17.335814 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b17bc80-850e-401d-a429-e82d4e2b093b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:17.343721 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 571/1157] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:57:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:17.349420 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09953985-ac7c-45a7-9cd5-6a56619a5b79 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:17.370777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:17.446674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09953985-ac7c-45a7-9cd5-6a56619a5b79 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:17.479677 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ab00f-5414-4030-8d00-a8db95e11762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:17.480560 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ab00f-5414-4030-8d00-a8db95e11762 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:17.480849 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ab00f-5414-4030-8d00-a8db95e11762 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:17.481127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ab00f-5414-4030-8d00-a8db95e11762 None admin] PATH_INFO: `/v3/users/0cac3d3a76a34aa790f5f4d6959d8a92` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:17.482291 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ab00f-5414-4030-8d00-a8db95e11762 None admin] RBAC: Authorizing `identity:get_user(user_id=0cac3d3a76a34aa790f5f4d6959d8a92)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:17.483722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ab00f-5414-4030-8d00-a8db95e11762 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:17.485356 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 587/1158] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:57:17 2022] GET /identity/v3/users/0cac3d3a76a34aa790f5f4d6959d8a92 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:17.490998 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17bc80-850e-401d-a429-e82d4e2b093b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:17.559359 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:17.591128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17bc80-850e-401d-a429-e82d4e2b093b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:17.622896 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd877e18-76fb-4668-9b08-8aecae7ff092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:17.623713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd877e18-76fb-4668-9b08-8aecae7ff092 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:17.624048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd877e18-76fb-4668-9b08-8aecae7ff092 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:17.624536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd877e18-76fb-4668-9b08-8aecae7ff092 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:17.690838 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd877e18-76fb-4668-9b08-8aecae7ff092 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:17.692831 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd877e18-76fb-4668-9b08-8aecae7ff092 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:17.759160 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:17.790635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 572/1159] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:57:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 13:57:17.796381 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ab00f-5414-4030-8d00-a8db95e11762 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:17.880764 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:17.892455 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ab00f-5414-4030-8d00-a8db95e11762 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:17.925069 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a4cc5e-3918-4219-95ae-aa756d776ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:17.926419 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4cc5e-3918-4219-95ae-aa756d776ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:17.926700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4cc5e-3918-4219-95ae-aa756d776ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:17.926995 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4cc5e-3918-4219-95ae-aa756d776ac4 None admin] PATH_INFO: `/v3/projects/5c6c6221968c40d38eb3255fb497ecc3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:17.928207 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a4cc5e-3918-4219-95ae-aa756d776ac4 None admin] RBAC: Authorizing `identity:get_project(project_id=5c6c6221968c40d38eb3255fb497ecc3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:17.929635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a4cc5e-3918-4219-95ae-aa756d776ac4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:17.931299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 588/1160] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:57:17 2022] GET /identity/v3/projects/5c6c6221968c40d38eb3255fb497ecc3 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:17.937338 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd877e18-76fb-4668-9b08-8aecae7ff092 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:18.034289 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd877e18-76fb-4668-9b08-8aecae7ff092 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:18.067372 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9f1ca-9263-45bc-8989-cf800afee32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:18.068046 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9f1ca-9263-45bc-8989-cf800afee32a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:18.068340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9f1ca-9263-45bc-8989-cf800afee32a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:18.068625 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9f1ca-9263-45bc-8989-cf800afee32a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:18.069453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b9f1ca-9263-45bc-8989-cf800afee32a None admin] RBAC: Adding query filter params (role.id=c60a99de086f45c6820f7872a6b43feb, scope.project.id=5c6c6221968c40d38eb3255fb497ecc3, user.id=0cac3d3a76a34aa790f5f4d6959d8a92) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:57:18.069770 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b9f1ca-9263-45bc-8989-cf800afee32a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:18.071193 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b9f1ca-9263-45bc-8989-cf800afee32a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:18.076868 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 573/1161] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:57:17 2022] GET /identity/v3/role_assignments?user.id=0cac3d3a76a34aa790f5f4d6959d8a92&scope.project.id=5c6c6221968c40d38eb3255fb497ecc3&role.id=c60a99de086f45c6820f7872a6b43feb => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:20.089301 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244400e-7eb6-4742-a96a-c7491a3c1885 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:20.089804 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244400e-7eb6-4742-a96a-c7491a3c1885 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:20.089916 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244400e-7eb6-4742-a96a-c7491a3c1885 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:20.091225 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 589/1162] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:57:20.096524 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369283b-8733-4c56-89bb-f0702736d05b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:20.096918 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369283b-8733-4c56-89bb-f0702736d05b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:20.097103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369283b-8733-4c56-89bb-f0702736d05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:20.113656 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-1369283b-8733-4c56-89bb-f0702736d05b None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:57:20.114837 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1369283b-8733-4c56-89bb-f0702736d05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:57:20.141320 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:20.182279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 574/1163] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:57:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:57:20.204808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244400e-7eb6-4742-a96a-c7491a3c1885 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:20.302913 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244400e-7eb6-4742-a96a-c7491a3c1885 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:20.335566 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d6334-0e1a-4b51-939a-cac4089b6e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:20.336225 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d6334-0e1a-4b51-939a-cac4089b6e44 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:20.336516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d6334-0e1a-4b51-939a-cac4089b6e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:20.336822 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d6334-0e1a-4b51-939a-cac4089b6e44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:20.359826 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:20.401802 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d6334-0e1a-4b51-939a-cac4089b6e44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:20.403247 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d6334-0e1a-4b51-939a-cac4089b6e44 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:20.496627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 590/1164] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:57:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 13:57:20.502467 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369283b-8733-4c56-89bb-f0702736d05b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:20.599914 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369283b-8733-4c56-89bb-f0702736d05b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:20.633284 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500cc1c-5fe3-4f25-89da-b2de68df4ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:20.634072 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500cc1c-5fe3-4f25-89da-b2de68df4ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:20.634362 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500cc1c-5fe3-4f25-89da-b2de68df4ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:20.634645 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500cc1c-5fe3-4f25-89da-b2de68df4ad7 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:20.641315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500cc1c-5fe3-4f25-89da-b2de68df4ad7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:20.642756 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500cc1c-5fe3-4f25-89da-b2de68df4ad7 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:20.656477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-3500cc1c-5fe3-4f25-89da-b2de68df4ad7 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 13:57:20.656477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:57:20.656477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 18 13:57:20.656477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 18 13:57:20.656477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:57:20.656477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:57:20.656477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:57:20.656477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:57:20.656477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 18 13:57:20.656477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:57:20.656477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:57:20.656477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:57:20.656477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:57:20.656477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:57:20.656477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:57:20.656477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:57:20.656477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:57:20.657683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:57:20.657683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:57:20.657683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:57:20.657683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:57:20.657683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:57:20.657683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:57:20.657683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:57:20.657683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:57:20.657683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:57:20.657683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:57:20.657683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:57:20.657683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:57:20.657683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:57:20.657683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:57:20.657683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:57:20.657683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:57:20.658811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:57:20.658811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:57:20.658811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:57:20.658811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:57:20.658811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:57:20.658811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:57:20.658811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:57:20.658811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:57:20.658811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:57:20.658811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:57:20.658811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 18 13:57:20.658811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:57:20.658811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 13:57:20.658811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:57:20.662897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 575/1165] 10.222.0.94 () {56 vars in 1117 bytes} [Mon Jul 18 13:57:20 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:57:20.663863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d6334-0e1a-4b51-939a-cac4089b6e44 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:20.717204 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:20.761549 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d6334-0e1a-4b51-939a-cac4089b6e44 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:20.793730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b323eb-d778-426b-97c8-9eb5eed6bae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:20.794385 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b323eb-d778-426b-97c8-9eb5eed6bae6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:20.794671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b323eb-d778-426b-97c8-9eb5eed6bae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:20.794948 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b323eb-d778-426b-97c8-9eb5eed6bae6 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:20.795666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b323eb-d778-426b-97c8-9eb5eed6bae6 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:57:20.795998 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b323eb-d778-426b-97c8-9eb5eed6bae6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:20.797360 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b323eb-d778-426b-97c8-9eb5eed6bae6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:20.804784 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 591/1166] 10.222.0.94 () {56 vars in 1115 bytes} [Mon Jul 18 13:57:20 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:57:22.771789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3df164-5c57-427b-b18a-4e5661eae114 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:22.772344 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3df164-5c57-427b-b18a-4e5661eae114 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:22.772437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3df164-5c57-427b-b18a-4e5661eae114 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:22.773736 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 576/1167] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:57:22.779447 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb033477-a884-4d2e-a541-79b89da05a3c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:22.779872 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb033477-a884-4d2e-a541-79b89da05a3c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:22.780031 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb033477-a884-4d2e-a541-79b89da05a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:22.797481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bb033477-a884-4d2e-a541-79b89da05a3c None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:57:22.798648 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bb033477-a884-4d2e-a541-79b89da05a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:57:22.862615 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 592/1168] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:57:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:57:22.884760 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3df164-5c57-427b-b18a-4e5661eae114 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:22.910306 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:22.985471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3df164-5c57-427b-b18a-4e5661eae114 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:23.018131 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d923d-3033-4edf-8492-fee4c434ec2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:23.019371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d923d-3033-4edf-8492-fee4c434ec2e None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:23.019662 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d923d-3033-4edf-8492-fee4c434ec2e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:23.019966 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d923d-3033-4edf-8492-fee4c434ec2e None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:23.020715 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d923d-3033-4edf-8492-fee4c434ec2e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:23.022138 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d923d-3033-4edf-8492-fee4c434ec2e None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:23.040214 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 577/1169] 10.222.0.94 () {60 vars in 1138 bytes} [Mon Jul 18 13:57:22 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:57:25.010817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4ac5d-dfcd-4938-81ed-5b5a5265d880 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:25.011303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4ac5d-dfcd-4938-81ed-5b5a5265d880 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:25.011396 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4ac5d-dfcd-4938-81ed-5b5a5265d880 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:25.012763 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 593/1170] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:57:25.017977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec7d3c-6240-4be5-a488-8018bf63af28 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:25.018357 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec7d3c-6240-4be5-a488-8018bf63af28 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:25.019044 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec7d3c-6240-4be5-a488-8018bf63af28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:25.035902 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-8eec7d3c-6240-4be5-a488-8018bf63af28 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:57:25.037109 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8eec7d3c-6240-4be5-a488-8018bf63af28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:57:25.101310 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 578/1171] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:57:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:57:25.123541 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4ac5d-dfcd-4938-81ed-5b5a5265d880 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:25.173237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:25.222766 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4ac5d-dfcd-4938-81ed-5b5a5265d880 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:25.255463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87935c1f-b8bd-4f3e-a679-ea5a68a51e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:25.256335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87935c1f-b8bd-4f3e-a679-ea5a68a51e32 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:25.256626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87935c1f-b8bd-4f3e-a679-ea5a68a51e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:25.256908 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87935c1f-b8bd-4f3e-a679-ea5a68a51e32 None admin] PATH_INFO: `/v3/domains/bb55f7d660904772adcae874bba7389d` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:25.267029 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87935c1f-b8bd-4f3e-a679-ea5a68a51e32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=bb55f7d660904772adcae874bba7389d)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:25.268538 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87935c1f-b8bd-4f3e-a679-ea5a68a51e32 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:25.270731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 594/1172] 10.222.0.94 () {56 vars in 1183 bytes} [Mon Jul 18 13:57:25 2022] GET /identity/v3/domains/bb55f7d660904772adcae874bba7389d => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:25.276528 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec7d3c-6240-4be5-a488-8018bf63af28 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:25.291066 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:25.372392 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec7d3c-6240-4be5-a488-8018bf63af28 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:25.405401 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d01f2-4d32-434d-b0a5-4d9d98e80271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:25.406051 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d01f2-4d32-434d-b0a5-4d9d98e80271 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:25.406339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d01f2-4d32-434d-b0a5-4d9d98e80271 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:25.406619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d01f2-4d32-434d-b0a5-4d9d98e80271 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:25.407331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d01f2-4d32-434d-b0a5-4d9d98e80271 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:25.408780 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d01f2-4d32-434d-b0a5-4d9d98e80271 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:25.433192 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 579/1173] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Jul 18 13:57:25 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:57:27.454901 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb6cc1-43e2-4a49-8373-aac1d1c328ab None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:27.455466 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb6cc1-43e2-4a49-8373-aac1d1c328ab None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:27.455466 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb6cc1-43e2-4a49-8373-aac1d1c328ab None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:27.456870 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 595/1174] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:57:27.462569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703c133-29d4-4aa1-815a-a6e47ca8e862 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:27.462976 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703c133-29d4-4aa1-815a-a6e47ca8e862 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:27.463220 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703c133-29d4-4aa1-815a-a6e47ca8e862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:27.489016 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-5703c133-29d4-4aa1-815a-a6e47ca8e862 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:57:27.490289 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5703c133-29d4-4aa1-815a-a6e47ca8e862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:57:27.548180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:27.557005 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 580/1175] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:57:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:57:27.581071 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb6cc1-43e2-4a49-8373-aac1d1c328ab None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:27.680041 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb6cc1-43e2-4a49-8373-aac1d1c328ab None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:27.689703 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:27.712721 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881b088-4660-41ed-9039-da1db46f063d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:27.713505 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881b088-4660-41ed-9039-da1db46f063d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:27.713795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881b088-4660-41ed-9039-da1db46f063d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:27.714070 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881b088-4660-41ed-9039-da1db46f063d None admin] PATH_INFO: `/v3/domains/bb55f7d660904772adcae874bba7389d` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:27.715233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881b088-4660-41ed-9039-da1db46f063d None admin] RBAC: Authorizing `identity:get_domain(domain_id=bb55f7d660904772adcae874bba7389d)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:27.716680 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881b088-4660-41ed-9039-da1db46f063d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:27.718260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 596/1176] 10.222.0.94 () {56 vars in 1183 bytes} [Mon Jul 18 13:57:27 2022] GET /identity/v3/domains/bb55f7d660904772adcae874bba7389d => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:27.724057 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703c133-29d4-4aa1-815a-a6e47ca8e862 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:27.819477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703c133-29d4-4aa1-815a-a6e47ca8e862 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:27.853886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca45b8-3b9b-41bf-a414-e6f1842e1d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:27.854551 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca45b8-3b9b-41bf-a414-e6f1842e1d08 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:27.854843 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca45b8-3b9b-41bf-a414-e6f1842e1d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:27.855191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca45b8-3b9b-41bf-a414-e6f1842e1d08 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:27.855907 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca45b8-3b9b-41bf-a414-e6f1842e1d08 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:27.857375 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca45b8-3b9b-41bf-a414-e6f1842e1d08 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:27.876779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 581/1177] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Jul 18 13:57:27 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:57:29.911714 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c159ae-2385-4a4d-a134-8d41030f3ed9 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:29.912329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c159ae-2385-4a4d-a134-8d41030f3ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:29.912428 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c159ae-2385-4a4d-a134-8d41030f3ed9 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:29.913893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 597/1178] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:57:29.918993 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606ef2d-2995-419a-8e58-bc86a009824e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:29.919373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606ef2d-2995-419a-8e58-bc86a009824e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:29.919560 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606ef2d-2995-419a-8e58-bc86a009824e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:29.936643 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-2606ef2d-2995-419a-8e58-bc86a009824e None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:57:29.937842 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2606ef2d-2995-419a-8e58-bc86a009824e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:57:29.984832 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:30.004971 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 582/1179] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:57:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:57:30.028107 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c159ae-2385-4a4d-a134-8d41030f3ed9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:30.130938 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c159ae-2385-4a4d-a134-8d41030f3ed9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:30.164614 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a3c03-d7aa-4550-9d27-a1a4fee917eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:30.165343 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a3c03-d7aa-4550-9d27-a1a4fee917eb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:30.165633 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a3c03-d7aa-4550-9d27-a1a4fee917eb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:30.165913 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a3c03-d7aa-4550-9d27-a1a4fee917eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:30.177084 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a3c03-d7aa-4550-9d27-a1a4fee917eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:57:30.185173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-966a3c03-d7aa-4550-9d27-a1a4fee917eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:57:30.185173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:57:30.185173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:57:30.185173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:57:30.185173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:57:30.185173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:57:30.185173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:57:30.185173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:57:30.185173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:57:30.185173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:57:30.185173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:57:30.185173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:57:30.185173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:57:30.185173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:57:30.185173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:57:30.185173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:57:30.186330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:57:30.186330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:57:30.186330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:57:30.186330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:57:30.186330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:57:30.186330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:57:30.186330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:57:30.186330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:57:30.186330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:57:30.186330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:57:30.186330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:57:30.186330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:57:30.186330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:57:30.186330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:57:30.186330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:57:30.186330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:57:30.187410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:57:30.187410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:57:30.187410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:57:30.187410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:57:30.187410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:57:30.187410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:57:30.187410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:57:30.187410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:57:30.187410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:57:30.187410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:57:30.187410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:57:30.187410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:57:30.187410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 598/1180] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:57:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:57:30.192823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606ef2d-2995-419a-8e58-bc86a009824e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:30.289048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606ef2d-2995-419a-8e58-bc86a009824e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:30.322201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec96d08-b23a-4e2e-b523-8ad7d70642b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:30.322860 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec96d08-b23a-4e2e-b523-8ad7d70642b4 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:30.323152 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec96d08-b23a-4e2e-b523-8ad7d70642b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:30.323438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec96d08-b23a-4e2e-b523-8ad7d70642b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:30.324217 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec96d08-b23a-4e2e-b523-8ad7d70642b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:57:30.324531 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec96d08-b23a-4e2e-b523-8ad7d70642b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:30.325908 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec96d08-b23a-4e2e-b523-8ad7d70642b4 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:30.334124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 583/1181] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:57:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:30.340159 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a3c03-d7aa-4550-9d27-a1a4fee917eb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:30.381384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:30.439211 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a3c03-d7aa-4550-9d27-a1a4fee917eb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:30.473397 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3758d-849b-4e66-b94f-719b68586e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:30.474258 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3758d-849b-4e66-b94f-719b68586e78 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:30.474548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3758d-849b-4e66-b94f-719b68586e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:30.474830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3758d-849b-4e66-b94f-719b68586e78 None admin] PATH_INFO: `/v3/users/1a214d7a7cb147ed83da438bf62fd575` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:30.487671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3758d-849b-4e66-b94f-719b68586e78 None admin] RBAC: Authorizing `identity:get_user(user_id=1a214d7a7cb147ed83da438bf62fd575)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:30.489144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3758d-849b-4e66-b94f-719b68586e78 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:30.490765 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 599/1182] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:57:30 2022] GET /identity/v3/users/1a214d7a7cb147ed83da438bf62fd575 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:30.496385 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec96d08-b23a-4e2e-b523-8ad7d70642b4 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:30.541314 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:30.594380 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec96d08-b23a-4e2e-b523-8ad7d70642b4 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:30.626416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c0832-1e2f-4774-a0c1-efad8c80bd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:30.627023 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c0832-1e2f-4774-a0c1-efad8c80bd36 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:30.627302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c0832-1e2f-4774-a0c1-efad8c80bd36 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:30.628134 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c0832-1e2f-4774-a0c1-efad8c80bd36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:30.692469 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c0832-1e2f-4774-a0c1-efad8c80bd36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:30.693858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c0832-1e2f-4774-a0c1-efad8c80bd36 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:30.736073 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:30.790465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 584/1183] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:57:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 13:57:30.796287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3758d-849b-4e66-b94f-719b68586e78 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:30.887126 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:30.893001 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3758d-849b-4e66-b94f-719b68586e78 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:30.925881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53334abc-8da3-4548-842d-a7a860e64929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:30.926729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53334abc-8da3-4548-842d-a7a860e64929 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:30.927012 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53334abc-8da3-4548-842d-a7a860e64929 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:30.927291 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53334abc-8da3-4548-842d-a7a860e64929 None admin] PATH_INFO: `/v3/projects/4274f90690e64128a39971fb66ce92a0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:30.928475 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53334abc-8da3-4548-842d-a7a860e64929 None admin] RBAC: Authorizing `identity:get_project(project_id=4274f90690e64128a39971fb66ce92a0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:30.929903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53334abc-8da3-4548-842d-a7a860e64929 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:30.931575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 600/1184] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:57:30 2022] GET /identity/v3/projects/4274f90690e64128a39971fb66ce92a0 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:30.938278 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c0832-1e2f-4774-a0c1-efad8c80bd36 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:31.035963 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c0832-1e2f-4774-a0c1-efad8c80bd36 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:31.068586 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555a933-07d2-44fe-a8a9-249a5ec61bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:31.069230 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555a933-07d2-44fe-a8a9-249a5ec61bbb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:31.069515 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555a933-07d2-44fe-a8a9-249a5ec61bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:31.069793 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555a933-07d2-44fe-a8a9-249a5ec61bbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:31.070609 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555a933-07d2-44fe-a8a9-249a5ec61bbb None admin] RBAC: Adding query filter params (role.id=c60a99de086f45c6820f7872a6b43feb, scope.project.id=4274f90690e64128a39971fb66ce92a0, user.id=1a214d7a7cb147ed83da438bf62fd575) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:57:31.070918 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555a933-07d2-44fe-a8a9-249a5ec61bbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:31.072357 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555a933-07d2-44fe-a8a9-249a5ec61bbb None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:31.077888 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 585/1185] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:57:30 2022] GET /identity/v3/role_assignments?user.id=1a214d7a7cb147ed83da438bf62fd575&scope.project.id=4274f90690e64128a39971fb66ce92a0&role.id=c60a99de086f45c6820f7872a6b43feb => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:33.040095 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb8027-1159-4625-88e6-40463089f6fb None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:33.040609 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb8027-1159-4625-88e6-40463089f6fb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:33.040726 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb8027-1159-4625-88e6-40463089f6fb None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:33.042088 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 601/1186] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:57:33.047299 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daece81-2f21-49cc-863d-5f974e163a9d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:33.047722 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daece81-2f21-49cc-863d-5f974e163a9d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:33.047862 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daece81-2f21-49cc-863d-5f974e163a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:33.065082 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-6daece81-2f21-49cc-863d-5f974e163a9d None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:57:33.066842 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6daece81-2f21-49cc-863d-5f974e163a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:57:33.068786 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:33.132665 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 586/1187] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:57:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:57:33.154295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb8027-1159-4625-88e6-40463089f6fb None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:33.252166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb8027-1159-4625-88e6-40463089f6fb None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:33.285035 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc3f4d-61bb-47db-92fb-a8be43e171a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:33.285812 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc3f4d-61bb-47db-92fb-a8be43e171a4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:33.286096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc3f4d-61bb-47db-92fb-a8be43e171a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:33.286372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc3f4d-61bb-47db-92fb-a8be43e171a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:33.296106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc3f4d-61bb-47db-92fb-a8be43e171a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:57:33.303584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-41fc3f4d-61bb-47db-92fb-a8be43e171a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:57:33.303584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:57:33.303584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:57:33.303584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:57:33.303584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:57:33.303584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:57:33.303584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:57:33.303584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:57:33.303584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:57:33.303584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:57:33.303584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:57:33.303584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:57:33.303584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:57:33.303584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:57:33.303584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:57:33.303584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:57:33.305079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:57:33.305079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:57:33.305079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:57:33.305079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:57:33.305079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:57:33.305079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:57:33.305079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:57:33.305079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:57:33.305079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:57:33.305079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:57:33.305079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:57:33.305079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:57:33.305079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:57:33.305079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:57:33.305079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:57:33.305079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:57:33.306322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:57:33.306322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:57:33.306322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:57:33.306322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:57:33.306322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:57:33.306322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:57:33.306322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:57:33.306322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:57:33.306322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:57:33.306322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:57:33.306322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:57:33.306322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:57:33.306322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 602/1188] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:57:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:57:33.310543 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daece81-2f21-49cc-863d-5f974e163a9d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:33.406387 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daece81-2f21-49cc-863d-5f974e163a9d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:33.439072 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6392d-48b0-4849-9bbd-ebbdaef96f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:33.439727 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6392d-48b0-4849-9bbd-ebbdaef96f96 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:33.440571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6392d-48b0-4849-9bbd-ebbdaef96f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:33.440852 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6392d-48b0-4849-9bbd-ebbdaef96f96 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:33.441595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b6392d-48b0-4849-9bbd-ebbdaef96f96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:57:33.441902 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b6392d-48b0-4849-9bbd-ebbdaef96f96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:33.443267 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b6392d-48b0-4849-9bbd-ebbdaef96f96 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:33.448123 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:33.452134 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 587/1189] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:57:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:33.457929 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc3f4d-61bb-47db-92fb-a8be43e171a4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:33.483550 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:33.560040 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc3f4d-61bb-47db-92fb-a8be43e171a4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:33.592835 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4438a47-c1b1-400f-ab7f-8c64beb311fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:33.593696 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4438a47-c1b1-400f-ab7f-8c64beb311fb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:33.593985 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4438a47-c1b1-400f-ab7f-8c64beb311fb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:33.594263 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4438a47-c1b1-400f-ab7f-8c64beb311fb None admin] PATH_INFO: `/v3/users/1a214d7a7cb147ed83da438bf62fd575` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:33.595441 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4438a47-c1b1-400f-ab7f-8c64beb311fb None admin] RBAC: Authorizing `identity:get_user(user_id=1a214d7a7cb147ed83da438bf62fd575)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:33.597540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4438a47-c1b1-400f-ab7f-8c64beb311fb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:33.599329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 603/1190] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:57:33 2022] GET /identity/v3/users/1a214d7a7cb147ed83da438bf62fd575 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:33.605031 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6392d-48b0-4849-9bbd-ebbdaef96f96 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:33.703522 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6392d-48b0-4849-9bbd-ebbdaef96f96 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:33.735281 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b9182-0f36-4486-b064-deefbb4b1393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:33.736155 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b9182-0f36-4486-b064-deefbb4b1393 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:33.736445 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b9182-0f36-4486-b064-deefbb4b1393 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:33.737264 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b9182-0f36-4486-b064-deefbb4b1393 None admin] PATH_INFO: `/v3/projects/4274f90690e64128a39971fb66ce92a0` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:33.738432 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b9182-0f36-4486-b064-deefbb4b1393 None admin] RBAC: Authorizing `identity:get_project(project_id=4274f90690e64128a39971fb66ce92a0)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:33.739871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b9182-0f36-4486-b064-deefbb4b1393 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:33.741575 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 588/1191] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:57:33 2022] GET /identity/v3/projects/4274f90690e64128a39971fb66ce92a0 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:57:33.747502 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4438a47-c1b1-400f-ab7f-8c64beb311fb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:33.837168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:33.846898 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4438a47-c1b1-400f-ab7f-8c64beb311fb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:33.878940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56662801-2598-4223-b78f-2d99fe8335b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:33.879816 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56662801-2598-4223-b78f-2d99fe8335b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:33.880131 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56662801-2598-4223-b78f-2d99fe8335b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:33.880418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56662801-2598-4223-b78f-2d99fe8335b4 None admin] PATH_INFO: `/v3/projects/4274f90690e64128a39971fb66ce92a0/users/1a214d7a7cb147ed83da438bf62fd575/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:33.882367 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56662801-2598-4223-b78f-2d99fe8335b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=4274f90690e64128a39971fb66ce92a0, user_id=1a214d7a7cb147ed83da438bf62fd575, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:33.883788 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56662801-2598-4223-b78f-2d99fe8335b4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:33.896389 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 604/1192] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:57:33 2022] PUT /identity/v3/projects/4274f90690e64128a39971fb66ce92a0/users/1a214d7a7cb147ed83da438bf62fd575/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:57:35.846739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57046b0f-b23c-4fbb-b9fb-d218890e2853 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:35.847379 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57046b0f-b23c-4fbb-b9fb-d218890e2853 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:35.847379 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57046b0f-b23c-4fbb-b9fb-d218890e2853 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:35.848856 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 589/1193] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:57:35.854015 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4bf03-82a1-463a-b24e-bd43e4409c9c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:35.854430 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4bf03-82a1-463a-b24e-bd43e4409c9c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:35.854584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4bf03-82a1-463a-b24e-bd43e4409c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:35.870899 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-12f4bf03-82a1-463a-b24e-bd43e4409c9c None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:57:35.872119 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-12f4bf03-82a1-463a-b24e-bd43e4409c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:57:35.936482 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 605/1194] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:57:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:57:35.969746 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57046b0f-b23c-4fbb-b9fb-d218890e2853 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:36.038675 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:36.071987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57046b0f-b23c-4fbb-b9fb-d218890e2853 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:36.105225 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f64e6d-6a0b-47ef-adc3-fcff06cc2b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:36.105956 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f64e6d-6a0b-47ef-adc3-fcff06cc2b3e None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:36.106244 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f64e6d-6a0b-47ef-adc3-fcff06cc2b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:36.106530 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f64e6d-6a0b-47ef-adc3-fcff06cc2b3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:36.116246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f64e6d-6a0b-47ef-adc3-fcff06cc2b3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:57:36.123751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-96f64e6d-6a0b-47ef-adc3-fcff06cc2b3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:57:36.123751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:57:36.123751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:57:36.123751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:57:36.123751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:57:36.123751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:57:36.123751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:57:36.123751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:57:36.123751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:57:36.123751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:57:36.123751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:57:36.123751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:57:36.123751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:57:36.123751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:57:36.123751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:57:36.123751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:57:36.124769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:57:36.124769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:57:36.124769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:57:36.124769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:57:36.124769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:57:36.124769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:57:36.124769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:57:36.124769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:57:36.124769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:57:36.124769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:57:36.124769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:57:36.124769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:57:36.124769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:57:36.124769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:57:36.124769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:57:36.124769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:57:36.125621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:57:36.125621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:57:36.125621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:57:36.125621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:57:36.125621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:57:36.125621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:57:36.125621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:57:36.125621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:57:36.125621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:57:36.125621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:57:36.125621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:57:36.125621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:57:36.125621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 590/1195] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:57:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:57:36.131120 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4bf03-82a1-463a-b24e-bd43e4409c9c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:36.208670 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:36.227719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4bf03-82a1-463a-b24e-bd43e4409c9c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:36.259574 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716aa257-5995-4170-a21a-6b234b6304d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:36.260265 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716aa257-5995-4170-a21a-6b234b6304d1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:36.260552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716aa257-5995-4170-a21a-6b234b6304d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:36.260829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716aa257-5995-4170-a21a-6b234b6304d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:36.261563 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716aa257-5995-4170-a21a-6b234b6304d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:57:36.261872 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716aa257-5995-4170-a21a-6b234b6304d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:36.263233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716aa257-5995-4170-a21a-6b234b6304d1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:36.271506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 606/1196] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:57:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:36.277049 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f64e6d-6a0b-47ef-adc3-fcff06cc2b3e None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:36.373415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f64e6d-6a0b-47ef-adc3-fcff06cc2b3e None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:36.399255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:36.407320 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c66c8e-382a-4290-9c05-236d6463c4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:36.408208 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c66c8e-382a-4290-9c05-236d6463c4db None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:36.408496 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c66c8e-382a-4290-9c05-236d6463c4db None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:36.408772 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c66c8e-382a-4290-9c05-236d6463c4db None admin] PATH_INFO: `/v3/users/1a214d7a7cb147ed83da438bf62fd575` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:36.409938 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c66c8e-382a-4290-9c05-236d6463c4db None admin] RBAC: Authorizing `identity:get_user(user_id=1a214d7a7cb147ed83da438bf62fd575)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:36.411362 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c66c8e-382a-4290-9c05-236d6463c4db None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:36.412986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 591/1197] 10.222.0.94 () {56 vars in 1177 bytes} [Mon Jul 18 13:57:36 2022] GET /identity/v3/users/1a214d7a7cb147ed83da438bf62fd575 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:57:36.418595 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716aa257-5995-4170-a21a-6b234b6304d1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:36.515754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716aa257-5995-4170-a21a-6b234b6304d1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:36.548084 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ba57d-fd55-4e83-b451-81797dda43df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:36.548713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ba57d-fd55-4e83-b451-81797dda43df None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:36.549231 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ba57d-fd55-4e83-b451-81797dda43df None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:36.549513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ba57d-fd55-4e83-b451-81797dda43df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:36.563145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:36.615257 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ba57d-fd55-4e83-b451-81797dda43df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:36.616726 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ba57d-fd55-4e83-b451-81797dda43df None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:36.711126 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 607/1198] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:57:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 13:57:36.716971 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c66c8e-382a-4290-9c05-236d6463c4db None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:36.813546 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c66c8e-382a-4290-9c05-236d6463c4db None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:36.846871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a952b3-7ae5-4529-b42b-4ab21d87ed63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:36.847718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a952b3-7ae5-4529-b42b-4ab21d87ed63 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:36.848051 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a952b3-7ae5-4529-b42b-4ab21d87ed63 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:36.848331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a952b3-7ae5-4529-b42b-4ab21d87ed63 None admin] PATH_INFO: `/v3/projects/4274f90690e64128a39971fb66ce92a0` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:36.849495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a952b3-7ae5-4529-b42b-4ab21d87ed63 None admin] RBAC: Authorizing `identity:get_project(project_id=4274f90690e64128a39971fb66ce92a0)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:36.850918 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a952b3-7ae5-4529-b42b-4ab21d87ed63 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:36.854335 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 592/1199] 10.222.0.94 () {56 vars in 1186 bytes} [Mon Jul 18 13:57:36 2022] GET /identity/v3/projects/4274f90690e64128a39971fb66ce92a0 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:36.860229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ba57d-fd55-4e83-b451-81797dda43df None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:57:36.904287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:36.957845 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ba57d-fd55-4e83-b451-81797dda43df None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:57:36.990757 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ca6a6-e591-465e-b2f7-7e73e6010294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:57:36.991336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ca6a6-e591-465e-b2f7-7e73e6010294 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:57:36.991624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ca6a6-e591-465e-b2f7-7e73e6010294 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:57:36.991905 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ca6a6-e591-465e-b2f7-7e73e6010294 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:57:36.992793 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ca6a6-e591-465e-b2f7-7e73e6010294 None admin] RBAC: Adding query filter params (role.id=c60a99de086f45c6820f7872a6b43feb, scope.project.id=4274f90690e64128a39971fb66ce92a0, user.id=1a214d7a7cb147ed83da438bf62fd575) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:57:36.993109 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ca6a6-e591-465e-b2f7-7e73e6010294 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:57:36.994539 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ca6a6-e591-465e-b2f7-7e73e6010294 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:57:37.000544 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 608/1200] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:57:36 2022] GET /identity/v3/role_assignments?user.id=1a214d7a7cb147ed83da438bf62fd575&scope.project.id=4274f90690e64128a39971fb66ce92a0&role.id=c60a99de086f45c6820f7872a6b43feb => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:57:46.842695 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=68 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:57:46.997768 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=46 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:31.743831 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c3528-7907-414d-8088-fe006fada471 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:31.744447 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c3528-7907-414d-8088-fe006fada471 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:31.744447 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c3528-7907-414d-8088-fe006fada471 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:31.746054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 593/1201] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:58:31.754557 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4d122-06ed-4430-8ceb-7d977aa16898 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:31.754557 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4d122-06ed-4430-8ceb-7d977aa16898 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:31.754557 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4d122-06ed-4430-8ceb-7d977aa16898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:31.771277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-afd4d122-06ed-4430-8ceb-7d977aa16898 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:58:31.772823 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-afd4d122-06ed-4430-8ceb-7d977aa16898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:58:31.858378 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 609/1202] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:58:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 13:58:31.890028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8c3528-7907-414d-8088-fe006fada471 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:32.032312 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8c3528-7907-414d-8088-fe006fada471 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:32.073891 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e81de-675f-4b90-9c56-be193fdde1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:32.074975 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e81de-675f-4b90-9c56-be193fdde1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:32.075267 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e81de-675f-4b90-9c56-be193fdde1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:32.075551 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e81de-675f-4b90-9c56-be193fdde1e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:32.089463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e81de-675f-4b90-9c56-be193fdde1e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:32.090988 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e81de-675f-4b90-9c56-be193fdde1e2 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:32.109671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-688e81de-675f-4b90-9c56-be193fdde1e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:32.109671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:32.109671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:32.109671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:32.109671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:32.109671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:32.109671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:32.109671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:32.109671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:32.109671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:32.109671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:32.109671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:32.109671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:58:32.109671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:58:32.109671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:58:32.110864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:58:32.110864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:32.110864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:32.110864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:32.110864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:32.110864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:32.110864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:32.110864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:32.110864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:32.110864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:32.110864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:32.110864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:32.110864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:32.110864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:32.110864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:32.110864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:32.110864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:32.111856 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:58:32.111856 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:58:32.111856 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:32.111856 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:58:32.111856 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 594/1203] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:58:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:58:32.117989 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4d122-06ed-4430-8ceb-7d977aa16898 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:32.253662 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4d122-06ed-4430-8ceb-7d977aa16898 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:32.291298 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:32.299064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c145be-7a6e-4eab-8439-abedf60f3322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:32.299821 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c145be-7a6e-4eab-8439-abedf60f3322 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:32.300162 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c145be-7a6e-4eab-8439-abedf60f3322 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:32.300470 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c145be-7a6e-4eab-8439-abedf60f3322 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:32.301300 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c145be-7a6e-4eab-8439-abedf60f3322 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:32.301637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c145be-7a6e-4eab-8439-abedf60f3322 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:32.303103 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c145be-7a6e-4eab-8439-abedf60f3322 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:32.317092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 610/1204] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:32.323214 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e81de-675f-4b90-9c56-be193fdde1e2 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:32.390964 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:32.459520 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e81de-675f-4b90-9c56-be193fdde1e2 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:32.510141 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd614031-4c3f-4789-bc1a-fa77a3cfa361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:32.510959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd614031-4c3f-4789-bc1a-fa77a3cfa361 None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:32.511401 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd614031-4c3f-4789-bc1a-fa77a3cfa361 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:32.511724 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd614031-4c3f-4789-bc1a-fa77a3cfa361 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:32.512651 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd614031-4c3f-4789-bc1a-fa77a3cfa361 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:32.515080 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd614031-4c3f-4789-bc1a-fa77a3cfa361 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:32.544965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 595/1205] 10.222.0.94 () {60 vars in 1132 bytes} [Mon Jul 18 13:58:32 2022] POST /identity/v3/users => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:58:34.842181 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864789c-f52f-46d7-81a3-0f52ed400e36 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:34.842730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864789c-f52f-46d7-81a3-0f52ed400e36 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:34.842730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864789c-f52f-46d7-81a3-0f52ed400e36 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:34.844314 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 611/1206] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:58:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:58:34.849700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5ff98-0fa7-4d29-945b-746f98442c8c None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:34.850165 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5ff98-0fa7-4d29-945b-746f98442c8c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:34.850256 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5ff98-0fa7-4d29-945b-746f98442c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:34.869698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-edf5ff98-0fa7-4d29-945b-746f98442c8c None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:58:34.871121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-edf5ff98-0fa7-4d29-945b-746f98442c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:58:34.961971 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 596/1207] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:58:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:58:34.976707 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864789c-f52f-46d7-81a3-0f52ed400e36 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:35.106012 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864789c-f52f-46d7-81a3-0f52ed400e36 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:35.145659 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e10ce-c498-4567-8323-0387416c5db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:35.148239 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e10ce-c498-4567-8323-0387416c5db0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:35.148598 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e10ce-c498-4567-8323-0387416c5db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:35.148890 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e10ce-c498-4567-8323-0387416c5db0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:35.162067 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e10ce-c498-4567-8323-0387416c5db0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:58:35.171537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c05e10ce-c498-4567-8323-0387416c5db0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:58:35.171537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:35.171537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:35.171537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:35.171537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:35.171537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:35.171537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:35.171537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:35.171537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:35.171537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:35.171537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:35.171537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:35.171537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:58:35.171537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:58:35.171537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:58:35.171537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:58:35.172642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:58:35.172642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:58:35.172642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:58:35.172642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:58:35.172642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:35.172642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:35.172642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:35.172642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:35.172642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:35.172642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:35.172642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:35.172642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:35.172642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:35.172642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:35.172642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:35.172642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:35.173756 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:35.173756 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:35.173756 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:35.173756 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:35.173756 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:58:35.173756 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:58:35.173756 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:58:35.173756 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:58:35.173756 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:58:35.173756 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:58:35.173756 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:58:35.173756 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:58:35.173756 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 612/1208] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:58:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:35.178736 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5ff98-0fa7-4d29-945b-746f98442c8c None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:35.222998 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:35.303973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5ff98-0fa7-4d29-945b-746f98442c8c None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:35.346643 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150fc52-3d84-4100-a975-461c4dc0b068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:35.347337 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150fc52-3d84-4100-a975-461c4dc0b068 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:35.347631 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150fc52-3d84-4100-a975-461c4dc0b068 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:35.347923 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150fc52-3d84-4100-a975-461c4dc0b068 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:35.348914 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2150fc52-3d84-4100-a975-461c4dc0b068 None admin] RBAC: Adding query filter params (name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:35.349222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2150fc52-3d84-4100-a975-461c4dc0b068 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:35.350861 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2150fc52-3d84-4100-a975-461c4dc0b068 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:35.361250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 597/1209] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Jul 18 13:58:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:35.367058 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e10ce-c498-4567-8323-0387416c5db0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:35.426391 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:35.491340 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e10ce-c498-4567-8323-0387416c5db0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:35.530267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33da24a-b0e5-4c6b-90b9-86e0276d1ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:35.531082 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33da24a-b0e5-4c6b-90b9-86e0276d1ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:35.531374 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33da24a-b0e5-4c6b-90b9-86e0276d1ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:35.531652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33da24a-b0e5-4c6b-90b9-86e0276d1ef0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:35.545788 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33da24a-b0e5-4c6b-90b9-86e0276d1ef0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:35.547406 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33da24a-b0e5-4c6b-90b9-86e0276d1ef0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:35.566405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c33da24a-b0e5-4c6b-90b9-86e0276d1ef0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:35.566405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:35.566405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:35.566405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:35.566405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:35.566405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:35.566405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:35.566405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:35.566405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:35.566405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:35.566405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:35.566405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:35.566405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:58:35.566405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:58:35.566405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:58:35.567554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:58:35.567554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:35.567554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:35.567554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:35.567554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:35.567554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:35.567554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:35.567554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:35.567554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:35.567554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:35.567554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:35.567554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:35.567554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:35.567554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:35.567554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:35.567554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:35.567554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:35.568792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:58:35.568792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:58:35.568792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:35.568792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:58:35.568792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 613/1210] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:58:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:35.573650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150fc52-3d84-4100-a975-461c4dc0b068 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:35.693553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:35.698116 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150fc52-3d84-4100-a975-461c4dc0b068 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:35.739067 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612164d-509d-4081-bcdc-5644f4493b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:35.739972 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612164d-509d-4081-bcdc-5644f4493b9b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:35.740275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612164d-509d-4081-bcdc-5644f4493b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:35.740733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612164d-509d-4081-bcdc-5644f4493b9b None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:35.741610 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612164d-509d-4081-bcdc-5644f4493b9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:35.741905 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612164d-509d-4081-bcdc-5644f4493b9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:35.743480 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612164d-509d-4081-bcdc-5644f4493b9b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:35.760398 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 598/1211] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:58:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:35.767034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33da24a-b0e5-4c6b-90b9-86e0276d1ef0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:35.885280 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33da24a-b0e5-4c6b-90b9-86e0276d1ef0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:35.887203 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:35.927071 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986f7a4-a6fa-486c-a8e5-6c183491a1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:35.928049 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986f7a4-a6fa-486c-a8e5-6c183491a1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:35.928357 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986f7a4-a6fa-486c-a8e5-6c183491a1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:35.928862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986f7a4-a6fa-486c-a8e5-6c183491a1b5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:35.938745 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986f7a4-a6fa-486c-a8e5-6c183491a1b5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:35.939417 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986f7a4-a6fa-486c-a8e5-6c183491a1b5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:35.956807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-d986f7a4-a6fa-486c-a8e5-6c183491a1b5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 13:58:35.956807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:35.956807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:35.956807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:35.956807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:35.956807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:35.956807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:35.956807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:35.956807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:35.956807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:35.956807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:35.956807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:35.956807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:58:35.956807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:58:35.956807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:58:35.956807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:58:35.958034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:35.958034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:35.958034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:58:35.958034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:58:35.958034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:58:35.958034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:58:35.958034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:35.958034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:35.958034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:35.958034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:35.958034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:35.958034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:35.958034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:35.958034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:35.958034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:35.958034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:35.958034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:35.958034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:35.959244 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:35.959244 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:35.959244 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:58:35.959244 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:58:35.959244 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:58:35.959244 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:58:35.959244 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:58:35.959244 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:58:35.959244 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 13:58:35.959244 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:58:35.959244 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 614/1212] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:58:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:35.964430 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612164d-509d-4081-bcdc-5644f4493b9b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:36.073176 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612164d-509d-4081-bcdc-5644f4493b9b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:36.112316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c03023-e06c-4bb0-8c9c-f61954f3e74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:36.113061 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c03023-e06c-4bb0-8c9c-f61954f3e74b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:36.113341 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c03023-e06c-4bb0-8c9c-f61954f3e74b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:36.113660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c03023-e06c-4bb0-8c9c-f61954f3e74b None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:36.114616 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c03023-e06c-4bb0-8c9c-f61954f3e74b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:36.114962 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c03023-e06c-4bb0-8c9c-f61954f3e74b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:36.116584 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c03023-e06c-4bb0-8c9c-f61954f3e74b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:36.135902 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 599/1213] 10.222.0.94 () {56 vars in 1143 bytes} [Mon Jul 18 13:58:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:36.142559 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986f7a4-a6fa-486c-a8e5-6c183491a1b5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:36.264672 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986f7a4-a6fa-486c-a8e5-6c183491a1b5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:36.300705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06b7d6-4f69-4ea7-9a61-c163b5e00cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:36.301409 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06b7d6-4f69-4ea7-9a61-c163b5e00cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:36.301777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06b7d6-4f69-4ea7-9a61-c163b5e00cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:36.302034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06b7d6-4f69-4ea7-9a61-c163b5e00cb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:36.335280 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:36.376967 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a06b7d6-4f69-4ea7-9a61-c163b5e00cb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:36.378619 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a06b7d6-4f69-4ea7-9a61-c163b5e00cb6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:36.497927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 615/1214] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:58:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 13:58:36.499321 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c03023-e06c-4bb0-8c9c-f61954f3e74b None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:36.521887 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:36.617224 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c03023-e06c-4bb0-8c9c-f61954f3e74b None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:36.658278 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f28e8-54cd-4623-b53f-3db230801b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:36.659349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f28e8-54cd-4623-b53f-3db230801b4a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:36.659499 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f28e8-54cd-4623-b53f-3db230801b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:36.659770 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f28e8-54cd-4623-b53f-3db230801b4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:36.672678 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f28e8-54cd-4623-b53f-3db230801b4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:36.674380 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f28e8-54cd-4623-b53f-3db230801b4a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:36.694254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-a34f28e8-54cd-4623-b53f-3db230801b4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:36.694254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:36.694254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:36.694254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:36.694254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:36.694254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:36.694254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:36.694254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:36.694254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:36.694254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:36.694254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:36.694254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:36.694254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:58:36.694254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:58:36.694254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:58:36.695393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:58:36.695393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:36.695393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:36.695393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:36.695393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:36.695393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:36.695393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:36.695393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:36.695393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:36.695393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:36.695393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:36.695393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:36.695393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:36.695393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:36.695393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:36.695393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:36.695393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:36.701341 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:58:36.701341 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:58:36.701341 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:36.701341 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:58:36.701341 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 600/1215] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:58:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:36.703090 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06b7d6-4f69-4ea7-9a61-c163b5e00cb6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:36.770257 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:36.819451 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06b7d6-4f69-4ea7-9a61-c163b5e00cb6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:36.858250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edb88a-eb44-483e-afd1-557b1c1a1576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:36.858919 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edb88a-eb44-483e-afd1-557b1c1a1576 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:36.859208 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edb88a-eb44-483e-afd1-557b1c1a1576 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:36.859487 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edb88a-eb44-483e-afd1-557b1c1a1576 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:36.860238 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6edb88a-eb44-483e-afd1-557b1c1a1576 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:36.860550 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6edb88a-eb44-483e-afd1-557b1c1a1576 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:36.862196 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6edb88a-eb44-483e-afd1-557b1c1a1576 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:36.880612 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 616/1216] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:36.882287 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f28e8-54cd-4623-b53f-3db230801b4a None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:36.972660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:37.006421 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f28e8-54cd-4623-b53f-3db230801b4a None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:37.051491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8839ee5-88b9-417e-b00a-e670d43275c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:37.052325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8839ee5-88b9-417e-b00a-e670d43275c7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:37.052610 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8839ee5-88b9-417e-b00a-e670d43275c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:37.053514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8839ee5-88b9-417e-b00a-e670d43275c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:37.062223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8839ee5-88b9-417e-b00a-e670d43275c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:37.063759 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8839ee5-88b9-417e-b00a-e670d43275c7 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:37.079901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-f8839ee5-88b9-417e-b00a-e670d43275c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:58:37.079901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:37.079901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:37.079901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:37.079901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:37.079901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:37.079901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:37.079901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:37.079901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:37.079901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:37.079901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:37.079901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:37.079901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:58:37.079901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:58:37.079901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:58:37.081472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:58:37.081472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:37.081472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:37.081472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:37.081472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:37.081472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:37.081472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:37.081472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:37.081472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:37.081472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:37.081472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:37.081472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:37.081472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:37.081472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:37.081472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:37.081472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:37.081472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:37.082619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:58:37.082619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:58:37.082619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:58:37.082619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:58:37.082619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:58:37.082619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:58:37.082619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:58:37.082619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:58:37.086060 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 601/1217] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:58:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:58:37.087329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edb88a-eb44-483e-afd1-557b1c1a1576 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:37.203746 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edb88a-eb44-483e-afd1-557b1c1a1576 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:37.229174 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:37.245659 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81def013-961d-491a-bbff-8390cf3e4b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:37.246333 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81def013-961d-491a-bbff-8390cf3e4b98 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:37.246620 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81def013-961d-491a-bbff-8390cf3e4b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:37.246908 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81def013-961d-491a-bbff-8390cf3e4b98 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:37.248270 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81def013-961d-491a-bbff-8390cf3e4b98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:37.248591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81def013-961d-491a-bbff-8390cf3e4b98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:37.250309 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81def013-961d-491a-bbff-8390cf3e4b98 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:37.263665 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 617/1218] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:58:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:37.269788 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8839ee5-88b9-417e-b00a-e670d43275c7 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:37.395206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8839ee5-88b9-417e-b00a-e670d43275c7 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:37.427109 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:37.433848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcee669-c612-4249-8e75-5087d51c5646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:37.434665 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcee669-c612-4249-8e75-5087d51c5646 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:37.435598 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcee669-c612-4249-8e75-5087d51c5646 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:37.436018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcee669-c612-4249-8e75-5087d51c5646 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:37.436958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcee669-c612-4249-8e75-5087d51c5646 None admin] RBAC: Adding query filter params (role.id=5c4ba827ac854bc69cc26c3bcd27dc5c, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=dd5b2ba9e37041c78a99f1700a4970c4) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:37.437380 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcee669-c612-4249-8e75-5087d51c5646 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:37.438889 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcee669-c612-4249-8e75-5087d51c5646 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:37.445444 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 602/1219] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:58:37 2022] GET /identity/v3/role_assignments?user.id=dd5b2ba9e37041c78a99f1700a4970c4&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=5c4ba827ac854bc69cc26c3bcd27dc5c => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:39.549507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54437c-f7f7-4924-9120-c7c1ec168991 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:39.550221 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54437c-f7f7-4924-9120-c7c1ec168991 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:39.550221 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54437c-f7f7-4924-9120-c7c1ec168991 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:39.551574 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 618/1220] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:58:39.556789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951cd0a-6c9c-472a-aa93-d5fc0d7416c5 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:39.557299 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951cd0a-6c9c-472a-aa93-d5fc0d7416c5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:39.557414 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951cd0a-6c9c-472a-aa93-d5fc0d7416c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:39.574870 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b951cd0a-6c9c-472a-aa93-d5fc0d7416c5 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:58:39.576707 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b951cd0a-6c9c-472a-aa93-d5fc0d7416c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:58:39.650910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 603/1221] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:58:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:58:39.681049 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54437c-f7f7-4924-9120-c7c1ec168991 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:39.790413 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54437c-f7f7-4924-9120-c7c1ec168991 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:39.831213 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b247c9-42ca-44d4-b620-bc3c8886ebb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:39.832008 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b247c9-42ca-44d4-b620-bc3c8886ebb4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:39.832373 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b247c9-42ca-44d4-b620-bc3c8886ebb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:39.832682 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b247c9-42ca-44d4-b620-bc3c8886ebb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:39.844496 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b247c9-42ca-44d4-b620-bc3c8886ebb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:58:39.852666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-01b247c9-42ca-44d4-b620-bc3c8886ebb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:58:39.852666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:39.852666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:39.852666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:39.852666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:39.852666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:39.852666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:39.852666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:39.852666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:39.852666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:39.852666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:39.852666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:39.852666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:58:39.852666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:58:39.852666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:58:39.852666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:58:39.854072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:58:39.854072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:58:39.854072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:58:39.854072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:58:39.854072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:39.854072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:39.854072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:39.854072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:39.854072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:39.854072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:39.854072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:39.854072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:39.854072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:39.854072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:39.854072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:39.854072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:39.855116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:39.855116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:39.855116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:39.855116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:39.855116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:58:39.855116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:58:39.855116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:58:39.855116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:58:39.855116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:58:39.855116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:58:39.855116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:58:39.855116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:58:39.855116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 619/1222] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:58:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:39.860503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951cd0a-6c9c-472a-aa93-d5fc0d7416c5 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:39.971155 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951cd0a-6c9c-472a-aa93-d5fc0d7416c5 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:39.996960 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:40.010297 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9bde1b-64d4-4042-8b82-0fc79eb1f3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:40.010964 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bde1b-64d4-4042-8b82-0fc79eb1f3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:40.011246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bde1b-64d4-4042-8b82-0fc79eb1f3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:40.011521 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bde1b-64d4-4042-8b82-0fc79eb1f3d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:40.012297 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9bde1b-64d4-4042-8b82-0fc79eb1f3d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:40.012613 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9bde1b-64d4-4042-8b82-0fc79eb1f3d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:40.014172 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9bde1b-64d4-4042-8b82-0fc79eb1f3d1 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:40.022206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 604/1223] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Jul 18 13:58:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:40.028166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b247c9-42ca-44d4-b620-bc3c8886ebb4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:40.074344 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:40.134302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b247c9-42ca-44d4-b620-bc3c8886ebb4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:40.170472 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362d72e-a46a-4874-951e-abafaf2b819a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:40.171333 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362d72e-a46a-4874-951e-abafaf2b819a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:40.171649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362d72e-a46a-4874-951e-abafaf2b819a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:40.171960 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362d72e-a46a-4874-951e-abafaf2b819a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:40.183873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362d72e-a46a-4874-951e-abafaf2b819a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:40.185602 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362d72e-a46a-4874-951e-abafaf2b819a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:40.203951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b362d72e-a46a-4874-951e-abafaf2b819a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:40.203951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:40.203951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:40.203951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:40.203951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:40.203951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:40.203951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:40.203951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:40.203951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:40.203951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:40.203951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:40.203951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:40.203951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:58:40.203951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:58:40.203951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:58:40.205092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:58:40.205092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:40.205092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:40.205092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:40.205092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:40.205092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:40.205092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:40.205092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:40.205092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:40.205092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:40.205092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:40.205092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:40.205092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:40.205092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:40.205092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:40.205092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:40.205092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:40.206034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:58:40.206034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:58:40.206034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:40.206034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:58:40.206034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 620/1224] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:58:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:40.212464 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9bde1b-64d4-4042-8b82-0fc79eb1f3d1 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:40.328124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9bde1b-64d4-4042-8b82-0fc79eb1f3d1 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:40.365742 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc9d61-4449-425f-a50e-1ed815cb663a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:40.366518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc9d61-4449-425f-a50e-1ed815cb663a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:40.366850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc9d61-4449-425f-a50e-1ed815cb663a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:40.367230 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc9d61-4449-425f-a50e-1ed815cb663a None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:40.368221 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc9d61-4449-425f-a50e-1ed815cb663a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:40.368536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc9d61-4449-425f-a50e-1ed815cb663a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:40.370180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc9d61-4449-425f-a50e-1ed815cb663a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:40.389878 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 605/1225] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:40.390575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362d72e-a46a-4874-951e-abafaf2b819a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:40.500065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:40.507313 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362d72e-a46a-4874-951e-abafaf2b819a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:40.553507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe873e22-f204-425c-9e3b-2e9c4e90d251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:40.554611 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe873e22-f204-425c-9e3b-2e9c4e90d251 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:40.554971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe873e22-f204-425c-9e3b-2e9c4e90d251 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:40.555257 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe873e22-f204-425c-9e3b-2e9c4e90d251 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:40.565387 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe873e22-f204-425c-9e3b-2e9c4e90d251 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:40.566877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe873e22-f204-425c-9e3b-2e9c4e90d251 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:40.584328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-fe873e22-f204-425c-9e3b-2e9c4e90d251 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 13:58:40.584328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:40.584328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:40.584328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:40.584328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:40.584328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:40.584328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:40.584328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:40.584328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:40.584328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:40.584328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:40.584328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:40.584328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:58:40.584328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:58:40.584328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:58:40.584328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:58:40.585640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:40.585640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:40.585640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:58:40.585640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:58:40.585640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:58:40.585640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:58:40.585640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:40.585640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:40.585640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:40.585640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:40.585640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:40.585640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:40.585640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:40.585640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:40.585640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:40.585640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:40.585640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:40.585640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:40.586820 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:40.586820 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:40.586820 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:58:40.586820 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:58:40.586820 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:58:40.586820 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:58:40.586820 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:58:40.586820 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:58:40.586820 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 13:58:40.586820 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:58:40.586820 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 621/1226] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:58:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:40.592712 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc9d61-4449-425f-a50e-1ed815cb663a None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:40.656297 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:40.730839 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc9d61-4449-425f-a50e-1ed815cb663a None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:40.769184 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c203db-df4a-4cd7-aa15-536741d93eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:40.769850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c203db-df4a-4cd7-aa15-536741d93eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:40.770136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c203db-df4a-4cd7-aa15-536741d93eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:40.770420 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c203db-df4a-4cd7-aa15-536741d93eb5 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:40.772231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c203db-df4a-4cd7-aa15-536741d93eb5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:40.772557 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c203db-df4a-4cd7-aa15-536741d93eb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:40.774008 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c203db-df4a-4cd7-aa15-536741d93eb5 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:40.792727 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 606/1227] 10.222.0.94 () {56 vars in 1143 bytes} [Mon Jul 18 13:58:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:40.799099 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe873e22-f204-425c-9e3b-2e9c4e90d251 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:40.903175 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe873e22-f204-425c-9e3b-2e9c4e90d251 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:40.942053 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72c223-dd34-498f-85bf-74a39da3f08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:40.942800 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72c223-dd34-498f-85bf-74a39da3f08f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:40.943085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72c223-dd34-498f-85bf-74a39da3f08f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:40.943366 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72c223-dd34-498f-85bf-74a39da3f08f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:40.956220 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af72c223-dd34-498f-85bf-74a39da3f08f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:40.957714 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af72c223-dd34-498f-85bf-74a39da3f08f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:40.975086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-af72c223-dd34-498f-85bf-74a39da3f08f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:40.975086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:40.975086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:40.975086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:40.975086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:40.975086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:40.975086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:40.975086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:40.975086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:40.975086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:40.975086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:40.975086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:40.975086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:58:40.975086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:58:40.975086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:58:40.976212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:58:40.976212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:40.976212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:40.976212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:40.976212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:40.976212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:40.976212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:40.976212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:40.976212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:40.976212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:40.976212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:40.976212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:40.976212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:40.976212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:40.976212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:40.976212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:40.976212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:40.977380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:58:40.977380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:58:40.977380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:40.977380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:58:40.981315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 622/1228] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:58:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:40.982875 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c203db-df4a-4cd7-aa15-536741d93eb5 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:41.086724 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c203db-df4a-4cd7-aa15-536741d93eb5 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:41.089049 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:41.123465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3faf1-9bb6-475e-9901-409b5b61a358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:41.124441 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3faf1-9bb6-475e-9901-409b5b61a358 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:41.124738 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3faf1-9bb6-475e-9901-409b5b61a358 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:41.125032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3faf1-9bb6-475e-9901-409b5b61a358 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:41.126097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3faf1-9bb6-475e-9901-409b5b61a358 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:41.126565 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3faf1-9bb6-475e-9901-409b5b61a358 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:41.128299 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3faf1-9bb6-475e-9901-409b5b61a358 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:41.142204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 607/1229] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:58:41.148344 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72c223-dd34-498f-85bf-74a39da3f08f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:41.160229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:41.277284 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72c223-dd34-498f-85bf-74a39da3f08f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:41.318037 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888227bc-0eda-4f30-8a32-fb0e7435845d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:41.318984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888227bc-0eda-4f30-8a32-fb0e7435845d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:41.319364 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888227bc-0eda-4f30-8a32-fb0e7435845d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:41.319719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888227bc-0eda-4f30-8a32-fb0e7435845d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:41.328167 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888227bc-0eda-4f30-8a32-fb0e7435845d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:41.329650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888227bc-0eda-4f30-8a32-fb0e7435845d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:41.344195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-888227bc-0eda-4f30-8a32-fb0e7435845d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:58:41.344195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:41.344195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:41.344195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:41.344195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:41.344195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:41.344195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:41.344195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:41.344195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:41.344195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:41.344195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:41.344195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:41.344195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:58:41.344195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:58:41.344195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:58:41.345351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:58:41.345351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:41.345351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:41.345351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:41.345351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:41.345351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:41.345351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:41.345351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:41.345351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:41.345351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:41.345351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:41.345351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:41.345351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:41.345351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:41.345351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:41.345351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:41.345351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:41.346300 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:58:41.346300 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:58:41.346300 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:58:41.346300 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:58:41.346300 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:58:41.346300 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:58:41.346300 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:58:41.346300 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:58:41.346300 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 623/1230] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:58:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:41.351210 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3faf1-9bb6-475e-9901-409b5b61a358 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:41.463330 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3faf1-9bb6-475e-9901-409b5b61a358 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:41.498427 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4d558-9132-4719-b5f2-e0eba0d7aa6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:41.499679 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4d558-9132-4719-b5f2-e0eba0d7aa6f None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:41.500013 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4d558-9132-4719-b5f2-e0eba0d7aa6f None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:41.500444 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4d558-9132-4719-b5f2-e0eba0d7aa6f None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:41.501247 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4d558-9132-4719-b5f2-e0eba0d7aa6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:41.501565 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4d558-9132-4719-b5f2-e0eba0d7aa6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:41.503524 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4d558-9132-4719-b5f2-e0eba0d7aa6f None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:41.518439 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 608/1231] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:58:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:41.524967 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888227bc-0eda-4f30-8a32-fb0e7435845d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:41.602524 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:41.642440 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888227bc-0eda-4f30-8a32-fb0e7435845d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:41.684035 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492eeb99-c657-4b36-9485-4747d8016b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:41.685447 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492eeb99-c657-4b36-9485-4747d8016b49 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:41.685733 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492eeb99-c657-4b36-9485-4747d8016b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:41.686009 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492eeb99-c657-4b36-9485-4747d8016b49 None admin] PATH_INFO: `/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/dd5b2ba9e37041c78a99f1700a4970c4/roles/5c4ba827ac854bc69cc26c3bcd27dc5c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:41.701688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492eeb99-c657-4b36-9485-4747d8016b49 None admin] RBAC: Authorizing `identity:create_grant(project_id=ec94946acf91486ba3f5b4fe834b8076, user_id=dd5b2ba9e37041c78a99f1700a4970c4, role_id=5c4ba827ac854bc69cc26c3bcd27dc5c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:41.703132 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492eeb99-c657-4b36-9485-4747d8016b49 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:41.715345 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 624/1232] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:58:41 2022] PUT /identity/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/dd5b2ba9e37041c78a99f1700a4970c4/roles/5c4ba827ac854bc69cc26c3bcd27dc5c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:58:43.799853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a586a0-09d3-41fb-bde0-1e46b1da9ed8 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:43.800491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a586a0-09d3-41fb-bde0-1e46b1da9ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:43.800584 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a586a0-09d3-41fb-bde0-1e46b1da9ed8 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:43.802117 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 609/1233] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:58:43.808480 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366c7a0-da31-4629-9411-28702e726c7f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:43.808982 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366c7a0-da31-4629-9411-28702e726c7f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:43.809060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366c7a0-da31-4629-9411-28702e726c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:43.828269 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b366c7a0-da31-4629-9411-28702e726c7f None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:58:43.829494 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b366c7a0-da31-4629-9411-28702e726c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:58:43.898716 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 625/1234] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:58:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:58:43.922323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a586a0-09d3-41fb-bde0-1e46b1da9ed8 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:43.951233 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:44.030967 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a586a0-09d3-41fb-bde0-1e46b1da9ed8 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:44.065869 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934d2c5-b205-4857-9332-9f19f3447001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:44.066686 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934d2c5-b205-4857-9332-9f19f3447001 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:44.067043 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934d2c5-b205-4857-9332-9f19f3447001 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:44.067394 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934d2c5-b205-4857-9332-9f19f3447001 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:44.080252 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934d2c5-b205-4857-9332-9f19f3447001 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:58:44.089462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-9934d2c5-b205-4857-9332-9f19f3447001 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:58:44.089462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:44.089462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:44.089462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:44.089462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:44.089462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:44.089462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:44.089462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:44.089462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:44.089462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:44.089462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:44.089462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:44.089462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:58:44.089462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:58:44.089462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:58:44.089462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:58:44.090472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:58:44.090472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:58:44.090472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:58:44.090472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:58:44.090472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:44.090472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:44.090472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:44.090472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:44.090472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:44.090472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:44.090472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:44.090472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:44.090472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:44.090472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:44.090472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:44.090472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:44.091481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:44.091481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:44.091481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:44.091481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:44.091481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:58:44.091481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:58:44.091481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:58:44.091481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:58:44.091481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:58:44.091481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:58:44.091481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 13:58:44.091481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:58:44.091481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 610/1235] 10.222.0.94 () {56 vars in 1102 bytes} [Mon Jul 18 13:58:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:44.098144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366c7a0-da31-4629-9411-28702e726c7f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:44.146437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:44.203673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366c7a0-da31-4629-9411-28702e726c7f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:44.238621 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334f7c1-6963-4a8a-9414-d3d7719cfd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:44.239299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334f7c1-6963-4a8a-9414-d3d7719cfd15 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:44.239582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334f7c1-6963-4a8a-9414-d3d7719cfd15 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:44.239861 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334f7c1-6963-4a8a-9414-d3d7719cfd15 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:44.240712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334f7c1-6963-4a8a-9414-d3d7719cfd15 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:44.241021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334f7c1-6963-4a8a-9414-d3d7719cfd15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:44.242505 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334f7c1-6963-4a8a-9414-d3d7719cfd15 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:44.251909 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 626/1236] 10.222.0.94 () {56 vars in 1103 bytes} [Mon Jul 18 13:58:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:44.258060 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934d2c5-b205-4857-9332-9f19f3447001 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:44.369535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:44.393161 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934d2c5-b205-4857-9332-9f19f3447001 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:44.444417 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdede6af-5050-4e04-8a91-e985cb2bf6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:44.445297 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdede6af-5050-4e04-8a91-e985cb2bf6dd None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:44.445585 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdede6af-5050-4e04-8a91-e985cb2bf6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:44.445872 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdede6af-5050-4e04-8a91-e985cb2bf6dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:44.457987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdede6af-5050-4e04-8a91-e985cb2bf6dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:44.459443 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdede6af-5050-4e04-8a91-e985cb2bf6dd None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:44.477808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-bdede6af-5050-4e04-8a91-e985cb2bf6dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:44.477808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:44.477808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:44.477808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:44.477808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:44.477808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:44.477808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:44.477808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:44.477808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:44.477808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:44.477808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:44.477808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:44.477808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:58:44.477808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:58:44.477808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:58:44.478933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:58:44.478933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:44.478933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:44.478933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:44.478933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:44.478933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:44.478933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:44.478933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:44.478933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:44.478933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:44.478933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:44.478933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:44.478933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:44.478933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:44.478933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:44.478933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:44.478933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:44.480238 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:58:44.480238 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:58:44.480238 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:44.480238 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:58:44.480238 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 611/1237] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:58:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:44.485542 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334f7c1-6963-4a8a-9414-d3d7719cfd15 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:44.595491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334f7c1-6963-4a8a-9414-d3d7719cfd15 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:44.605891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:44.643354 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48965ac-cee4-4743-ae7d-abfd68d7a101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:44.644166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48965ac-cee4-4743-ae7d-abfd68d7a101 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:44.644495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48965ac-cee4-4743-ae7d-abfd68d7a101 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:44.645040 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48965ac-cee4-4743-ae7d-abfd68d7a101 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:44.645791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48965ac-cee4-4743-ae7d-abfd68d7a101 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:44.646234 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48965ac-cee4-4743-ae7d-abfd68d7a101 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:44.648236 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48965ac-cee4-4743-ae7d-abfd68d7a101 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:44.664441 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 627/1238] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:58:44.670571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdede6af-5050-4e04-8a91-e985cb2bf6dd None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:44.794583 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdede6af-5050-4e04-8a91-e985cb2bf6dd None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:44.833782 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3cf3b9-3d0b-474e-a167-bab95a5a1bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:44.834616 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cf3b9-3d0b-474e-a167-bab95a5a1bee None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:44.834903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cf3b9-3d0b-474e-a167-bab95a5a1bee None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:44.835183 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cf3b9-3d0b-474e-a167-bab95a5a1bee None admin] PATH_INFO: `/v3/users/placement` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:44.838017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:44.844377 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3cf3b9-3d0b-474e-a167-bab95a5a1bee None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:44.846216 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3cf3b9-3d0b-474e-a167-bab95a5a1bee None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:44.861538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-bd3cf3b9-3d0b-474e-a167-bab95a5a1bee None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 13:58:44.861538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:44.861538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:44.861538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:44.861538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:44.861538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:44.861538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:44.861538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:44.861538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:44.861538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:44.861538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:44.861538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:44.861538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:58:44.861538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:58:44.861538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:58:44.861538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:58:44.862569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:44.862569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:44.862569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:58:44.862569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:58:44.862569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:58:44.862569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:58:44.862569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:44.862569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:44.862569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:44.862569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:44.862569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:44.862569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:44.862569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:44.862569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:44.862569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:44.862569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:44.862569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:44.862569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:44.864245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:44.864245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:44.864245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:58:44.864245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:58:44.864245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:58:44.864245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:58:44.864245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:58:44.864245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:58:44.864245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 13:58:44.864245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:58:44.864245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 612/1239] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:58:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:44.868595 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48965ac-cee4-4743-ae7d-abfd68d7a101 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:44.994628 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48965ac-cee4-4743-ae7d-abfd68d7a101 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:45.037625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cdb854-392a-4639-893f-edc5e2680c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:45.038304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cdb854-392a-4639-893f-edc5e2680c37 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:45.038594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cdb854-392a-4639-893f-edc5e2680c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:45.038873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cdb854-392a-4639-893f-edc5e2680c37 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:45.039722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cdb854-392a-4639-893f-edc5e2680c37 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:45.040077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cdb854-392a-4639-893f-edc5e2680c37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:45.042027 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cdb854-392a-4639-893f-edc5e2680c37 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:45.071247 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 628/1240] 10.222.0.94 () {56 vars in 1143 bytes} [Mon Jul 18 13:58:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:45.072626 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3cf3b9-3d0b-474e-a167-bab95a5a1bee None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:45.190206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3cf3b9-3d0b-474e-a167-bab95a5a1bee None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:45.229068 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e78cba-40f8-4534-b91e-8149e606911a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:45.229657 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e78cba-40f8-4534-b91e-8149e606911a None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:45.229944 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e78cba-40f8-4534-b91e-8149e606911a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:45.230229 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e78cba-40f8-4534-b91e-8149e606911a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:45.299181 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:45.306188 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e78cba-40f8-4534-b91e-8149e606911a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:45.307641 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e78cba-40f8-4534-b91e-8149e606911a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:45.426159 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 613/1241] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:58:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 13:58:45.432862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cdb854-392a-4639-893f-edc5e2680c37 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:45.464851 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:45.546666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cdb854-392a-4639-893f-edc5e2680c37 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:45.583693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990abe8-90f3-4235-8a14-b241b4d1f089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:45.584654 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990abe8-90f3-4235-8a14-b241b4d1f089 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:45.585022 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990abe8-90f3-4235-8a14-b241b4d1f089 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:45.585318 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990abe8-90f3-4235-8a14-b241b4d1f089 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:45.598317 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990abe8-90f3-4235-8a14-b241b4d1f089 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:45.600051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990abe8-90f3-4235-8a14-b241b4d1f089 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:45.619637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3990abe8-90f3-4235-8a14-b241b4d1f089 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:45.619637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:45.619637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:45.619637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:45.619637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:45.619637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:45.619637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:45.619637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:45.619637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:45.619637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:45.619637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:45.619637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:45.619637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:58:45.619637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:58:45.619637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:58:45.620984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:58:45.620984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:45.620984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:45.620984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:45.620984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:45.620984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:45.620984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:45.620984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:45.620984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:45.620984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:45.620984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:45.620984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:45.620984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:45.620984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:45.620984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:45.620984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:45.620984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:45.622212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:58:45.622212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:58:45.622212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:45.622212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:58:45.622212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 629/1242] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:58:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:58:45.627138 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e78cba-40f8-4534-b91e-8149e606911a None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:45.746073 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:45.761651 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e78cba-40f8-4534-b91e-8149e606911a None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:45.801137 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2750e-27ff-4ab9-870f-dff0ef75ed17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:45.801694 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2750e-27ff-4ab9-870f-dff0ef75ed17 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:45.802003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2750e-27ff-4ab9-870f-dff0ef75ed17 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:45.802256 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2750e-27ff-4ab9-870f-dff0ef75ed17 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:45.802976 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f2750e-27ff-4ab9-870f-dff0ef75ed17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:45.803279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f2750e-27ff-4ab9-870f-dff0ef75ed17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:45.804713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f2750e-27ff-4ab9-870f-dff0ef75ed17 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:45.816466 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 614/1243] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:58:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:45.822480 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990abe8-90f3-4235-8a14-b241b4d1f089 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:45.913384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:45.942571 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990abe8-90f3-4235-8a14-b241b4d1f089 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:45.987288 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df698d-8a6b-4cc5-8c28-6e5f3d63c77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:45.988363 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df698d-8a6b-4cc5-8c28-6e5f3d63c77b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:45.988827 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df698d-8a6b-4cc5-8c28-6e5f3d63c77b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:45.989245 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df698d-8a6b-4cc5-8c28-6e5f3d63c77b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:45.999236 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df698d-8a6b-4cc5-8c28-6e5f3d63c77b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:46.000908 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df698d-8a6b-4cc5-8c28-6e5f3d63c77b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:46.018358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-59df698d-8a6b-4cc5-8c28-6e5f3d63c77b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:58:46.018358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:46.018358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:46.018358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:46.018358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:46.018358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:46.018358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:46.018358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:46.018358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:46.018358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:46.018358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:46.018358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:46.018358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:58:46.018358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:58:46.018358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:58:46.019517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:58:46.019517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:46.019517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:46.019517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:46.019517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:46.019517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:46.019517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:46.019517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:46.019517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:46.019517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:46.019517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:46.019517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:46.019517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:46.019517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:46.019517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:46.019517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:46.019517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:46.020507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:58:46.020507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:58:46.020507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:58:46.020507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:58:46.020507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:58:46.020507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:58:46.020507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:58:46.020507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:58:46.020507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 630/1244] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:58:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:58:46.025877 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2750e-27ff-4ab9-870f-dff0ef75ed17 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:46.144861 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2750e-27ff-4ab9-870f-dff0ef75ed17 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:46.185938 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9e8be-c64d-4fd3-b7b4-51b73d97aeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:46.187137 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9e8be-c64d-4fd3-b7b4-51b73d97aeeb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:46.187725 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9e8be-c64d-4fd3-b7b4-51b73d97aeeb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:46.188212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9e8be-c64d-4fd3-b7b4-51b73d97aeeb None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:46.189251 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e9e8be-c64d-4fd3-b7b4-51b73d97aeeb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:46.189897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e9e8be-c64d-4fd3-b7b4-51b73d97aeeb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:46.191904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e9e8be-c64d-4fd3-b7b4-51b73d97aeeb None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:46.208869 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 615/1245] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:58:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:46.215176 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df698d-8a6b-4cc5-8c28-6e5f3d63c77b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:46.323010 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df698d-8a6b-4cc5-8c28-6e5f3d63c77b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:46.361509 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34646e-45a5-4a3c-b4a4-5bde105f0b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:46.362174 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34646e-45a5-4a3c-b4a4-5bde105f0b91 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:46.362462 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34646e-45a5-4a3c-b4a4-5bde105f0b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:46.362759 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34646e-45a5-4a3c-b4a4-5bde105f0b91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:46.363588 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34646e-45a5-4a3c-b4a4-5bde105f0b91 None admin] RBAC: Adding query filter params (role.id=5c4ba827ac854bc69cc26c3bcd27dc5c, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=dd5b2ba9e37041c78a99f1700a4970c4) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:46.363900 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34646e-45a5-4a3c-b4a4-5bde105f0b91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:46.365361 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34646e-45a5-4a3c-b4a4-5bde105f0b91 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:46.371089 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 631/1246] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:58:46 2022] GET /identity/v3/role_assignments?user.id=dd5b2ba9e37041c78a99f1700a4970c4&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=5c4ba827ac854bc69cc26c3bcd27dc5c => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:48.654899 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21cb0e-e4e3-424d-96a4-bce9105746da None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:48.655497 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21cb0e-e4e3-424d-96a4-bce9105746da None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:48.655497 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21cb0e-e4e3-424d-96a4-bce9105746da None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:48.657261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 616/1247] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:58:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 13:58:48.663202 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf195b-1d35-49ab-901b-0ccb819e3cfc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:48.663863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf195b-1d35-49ab-901b-0ccb819e3cfc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:48.663863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf195b-1d35-49ab-901b-0ccb819e3cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:48.669622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:48.682987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d7cf195b-1d35-49ab-901b-0ccb819e3cfc None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:58:48.684906 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d7cf195b-1d35-49ab-901b-0ccb819e3cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:58:48.766946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 632/1248] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:58:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:58:48.791769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21cb0e-e4e3-424d-96a4-bce9105746da None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:48.814682 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:48.920452 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21cb0e-e4e3-424d-96a4-bce9105746da None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:48.964327 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfc538-0f2f-4d23-a0a9-affda746d128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:48.966512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfc538-0f2f-4d23-a0a9-affda746d128 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:48.966811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfc538-0f2f-4d23-a0a9-affda746d128 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:48.967093 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfc538-0f2f-4d23-a0a9-affda746d128 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:48.979398 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecfc538-0f2f-4d23-a0a9-affda746d128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:58:48.989018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-9ecfc538-0f2f-4d23-a0a9-affda746d128 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:58:48.989018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:48.989018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:48.989018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:48.989018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:48.989018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:48.989018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:48.989018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:48.989018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:48.989018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:48.989018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:48.989018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:48.989018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:58:48.989018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:58:48.989018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:58:48.989018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:58:48.990163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:58:48.990163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:58:48.990163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:58:48.990163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:58:48.990163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:48.990163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:48.990163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:48.990163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:48.990163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:48.990163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:48.990163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:48.990163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:48.990163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:48.990163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:48.990163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:48.990163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:48.991032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:48.991032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:48.991032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:48.991032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:48.991032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:58:48.991032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:58:48.991032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:58:48.991032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:58:48.991032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:58:48.991032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:58:48.991032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:58:48.991032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:58:48.991032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 617/1249] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:58:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 13:58:48.998386 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf195b-1d35-49ab-901b-0ccb819e3cfc None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:49.099700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf195b-1d35-49ab-901b-0ccb819e3cfc None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:49.130797 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:49.136459 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553271f-20b2-43af-9073-35676397c1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:49.137978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553271f-20b2-43af-9073-35676397c1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:49.138290 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553271f-20b2-43af-9073-35676397c1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:49.138563 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553271f-20b2-43af-9073-35676397c1f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:49.139330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6553271f-20b2-43af-9073-35676397c1f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:49.139727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6553271f-20b2-43af-9073-35676397c1f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:49.141146 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6553271f-20b2-43af-9073-35676397c1f0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:49.149822 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 633/1250] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:58:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:58:49.155958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfc538-0f2f-4d23-a0a9-affda746d128 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:49.245017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:49.270803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfc538-0f2f-4d23-a0a9-affda746d128 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:49.305989 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d1be6-8617-4c7a-8d90-5aa18436e1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:49.306793 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d1be6-8617-4c7a-8d90-5aa18436e1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:49.307141 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d1be6-8617-4c7a-8d90-5aa18436e1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:49.307407 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4d1be6-8617-4c7a-8d90-5aa18436e1d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:49.320787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d1be6-8617-4c7a-8d90-5aa18436e1d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:49.322250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4d1be6-8617-4c7a-8d90-5aa18436e1d7 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:49.341848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-0a4d1be6-8617-4c7a-8d90-5aa18436e1d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:49.341848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:49.341848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:49.341848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:49.341848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:49.341848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:49.341848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:49.341848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:49.341848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:49.341848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:49.341848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:49.341848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:49.341848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:58:49.341848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:58:49.341848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:58:49.342946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:58:49.342946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:49.342946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:49.342946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:49.342946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:49.342946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:49.342946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:49.342946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:49.342946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:49.342946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:49.342946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:49.342946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:49.342946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:49.342946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:49.342946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:49.342946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:49.342946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:49.344289 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:58:49.344289 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:58:49.344289 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:49.344289 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:58:49.344289 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 618/1251] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:58:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:49.349312 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553271f-20b2-43af-9073-35676397c1f0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:49.468778 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553271f-20b2-43af-9073-35676397c1f0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:49.505876 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10397dfb-b46d-4688-af71-4b95ad23b5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:49.506819 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10397dfb-b46d-4688-af71-4b95ad23b5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:49.507258 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10397dfb-b46d-4688-af71-4b95ad23b5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:49.507678 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10397dfb-b46d-4688-af71-4b95ad23b5d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:49.508724 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10397dfb-b46d-4688-af71-4b95ad23b5d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:49.509275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10397dfb-b46d-4688-af71-4b95ad23b5d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:49.511586 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10397dfb-b46d-4688-af71-4b95ad23b5d6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:49.529439 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 634/1252] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:49.535472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d1be6-8617-4c7a-8d90-5aa18436e1d7 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:49.683185 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4d1be6-8617-4c7a-8d90-5aa18436e1d7 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:49.721240 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:49.731606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71ee65-e6d8-4b7d-9c7b-a19b1d201f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:49.732445 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71ee65-e6d8-4b7d-9c7b-a19b1d201f00 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:49.732734 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71ee65-e6d8-4b7d-9c7b-a19b1d201f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:49.733078 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71ee65-e6d8-4b7d-9c7b-a19b1d201f00 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:49.741684 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71ee65-e6d8-4b7d-9c7b-a19b1d201f00 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:49.743148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71ee65-e6d8-4b7d-9c7b-a19b1d201f00 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:49.758549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-ec71ee65-e6d8-4b7d-9c7b-a19b1d201f00 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 13:58:49.758549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:49.758549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:49.758549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:49.758549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:49.758549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:49.758549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:49.758549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:49.758549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:49.758549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:49.758549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:49.758549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:49.758549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:58:49.758549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:58:49.758549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:58:49.758549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:58:49.759660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:49.759660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:49.759660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:58:49.759660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:58:49.759660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:58:49.759660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:58:49.759660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:49.759660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:49.759660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:49.759660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:49.759660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:49.759660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:49.759660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:49.759660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:49.759660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:49.759660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:49.759660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:49.759660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:49.761122 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:49.761122 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:49.761122 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:58:49.761122 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:58:49.761122 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:58:49.761122 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:58:49.761122 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:58:49.761122 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:58:49.761122 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 13:58:49.761122 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:58:49.761122 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 619/1253] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:58:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:49.765985 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10397dfb-b46d-4688-af71-4b95ad23b5d6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:49.841678 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:49.890801 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10397dfb-b46d-4688-af71-4b95ad23b5d6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:49.928145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c51fe-e8e6-407b-b850-b8b1a12b8ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:49.928928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c51fe-e8e6-407b-b850-b8b1a12b8ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:49.929316 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c51fe-e8e6-407b-b850-b8b1a12b8ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:49.929605 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c51fe-e8e6-407b-b850-b8b1a12b8ec1 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:49.930487 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c51fe-e8e6-407b-b850-b8b1a12b8ec1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:49.930839 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c51fe-e8e6-407b-b850-b8b1a12b8ec1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:49.932386 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c51fe-e8e6-407b-b850-b8b1a12b8ec1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:49.955143 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 635/1254] 10.222.0.94 () {56 vars in 1143 bytes} [Mon Jul 18 13:58:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:58:49.960994 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71ee65-e6d8-4b7d-9c7b-a19b1d201f00 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:50.069265 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71ee65-e6d8-4b7d-9c7b-a19b1d201f00 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:50.104961 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd085b5b-2f9a-4c78-b693-2245afbc6272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:50.105642 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd085b5b-2f9a-4c78-b693-2245afbc6272 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:50.105930 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd085b5b-2f9a-4c78-b693-2245afbc6272 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:50.106209 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd085b5b-2f9a-4c78-b693-2245afbc6272 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:50.167867 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:50.182640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd085b5b-2f9a-4c78-b693-2245afbc6272 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:50.185387 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd085b5b-2f9a-4c78-b693-2245afbc6272 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:50.293635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 620/1255] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:58:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 13:58:50.299934 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c51fe-e8e6-407b-b850-b8b1a12b8ec1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:50.410797 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c51fe-e8e6-407b-b850-b8b1a12b8ec1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:50.425288 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:50.453589 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051cb5c0-47d0-4dd8-b0a6-1e208aef6423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:50.454808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051cb5c0-47d0-4dd8-b0a6-1e208aef6423 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:50.455202 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051cb5c0-47d0-4dd8-b0a6-1e208aef6423 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:50.455714 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051cb5c0-47d0-4dd8-b0a6-1e208aef6423 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:50.469629 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051cb5c0-47d0-4dd8-b0a6-1e208aef6423 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:50.471239 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051cb5c0-47d0-4dd8-b0a6-1e208aef6423 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:50.492507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-051cb5c0-47d0-4dd8-b0a6-1e208aef6423 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:50.492507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:50.492507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:50.492507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:50.492507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:50.492507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:50.492507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:50.492507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:50.492507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:50.492507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:50.492507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:50.492507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:50.492507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:58:50.492507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:58:50.492507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:58:50.493556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:58:50.493556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:50.493556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:50.493556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:50.493556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:50.493556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:50.493556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:50.493556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:50.493556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:50.493556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:50.493556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:50.493556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:50.493556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:50.493556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:50.493556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:50.493556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:50.493556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:50.499671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:58:50.499671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:58:50.499671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:50.499671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:58:50.499671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 636/1256] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:58:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:50.500019 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd085b5b-2f9a-4c78-b693-2245afbc6272 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:50.599030 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:50.612843 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd085b5b-2f9a-4c78-b693-2245afbc6272 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:50.655925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216557b-ce09-4c48-96f2-75258a679412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:50.656650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216557b-ce09-4c48-96f2-75258a679412 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:50.656987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216557b-ce09-4c48-96f2-75258a679412 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:50.657343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216557b-ce09-4c48-96f2-75258a679412 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:50.658087 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3216557b-ce09-4c48-96f2-75258a679412 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:50.658499 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3216557b-ce09-4c48-96f2-75258a679412 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:50.660029 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3216557b-ce09-4c48-96f2-75258a679412 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:50.675529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 621/1257] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:50.681324 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051cb5c0-47d0-4dd8-b0a6-1e208aef6423 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:50.791172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051cb5c0-47d0-4dd8-b0a6-1e208aef6423 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:50.826986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde5c94-7f44-49a5-b040-ef3de1106c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:50.827792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde5c94-7f44-49a5-b040-ef3de1106c8f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:50.828110 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde5c94-7f44-49a5-b040-ef3de1106c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:50.828387 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde5c94-7f44-49a5-b040-ef3de1106c8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:50.836020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdde5c94-7f44-49a5-b040-ef3de1106c8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:50.837488 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdde5c94-7f44-49a5-b040-ef3de1106c8f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:50.852059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-fdde5c94-7f44-49a5-b040-ef3de1106c8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:58:50.852059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:50.852059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:50.852059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:50.852059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:50.852059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:50.852059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:50.852059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:50.852059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:50.852059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:50.852059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:50.852059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:50.852059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:58:50.852059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:58:50.852059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:58:50.853138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:58:50.853138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:50.853138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:50.853138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:50.853138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:50.853138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:50.853138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:50.853138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:50.853138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:50.853138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:50.853138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:50.853138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:50.853138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:50.853138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:50.853138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:50.853138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:50.853138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:50.854050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:58:50.854050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:58:50.854050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:58:50.854050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:58:50.854050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:58:50.854050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:58:50.854050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:58:50.854050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:58:50.854050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 637/1258] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:58:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:50.860020 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216557b-ce09-4c48-96f2-75258a679412 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:50.973161 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216557b-ce09-4c48-96f2-75258a679412 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:51.013776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af48903-e161-4f70-9b03-b95fb3b06be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:51.014453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af48903-e161-4f70-9b03-b95fb3b06be3 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:51.014741 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af48903-e161-4f70-9b03-b95fb3b06be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:51.015021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af48903-e161-4f70-9b03-b95fb3b06be3 None admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:51.015832 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af48903-e161-4f70-9b03-b95fb3b06be3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:51.016177 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af48903-e161-4f70-9b03-b95fb3b06be3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:51.017635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af48903-e161-4f70-9b03-b95fb3b06be3 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:51.030977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 622/1259] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:58:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:51.037182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde5c94-7f44-49a5-b040-ef3de1106c8f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:51.063306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:51.141012 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde5c94-7f44-49a5-b040-ef3de1106c8f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:51.174504 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229b5dc-26a5-4656-969d-028d2f43c63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:51.175161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229b5dc-26a5-4656-969d-028d2f43c63a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:51.175448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229b5dc-26a5-4656-969d-028d2f43c63a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:51.175725 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229b5dc-26a5-4656-969d-028d2f43c63a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:51.176606 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7229b5dc-26a5-4656-969d-028d2f43c63a None admin] RBAC: Adding query filter params (role.id=c60a99de086f45c6820f7872a6b43feb, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=dd5b2ba9e37041c78a99f1700a4970c4) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:51.176917 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7229b5dc-26a5-4656-969d-028d2f43c63a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:51.178343 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7229b5dc-26a5-4656-969d-028d2f43c63a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:51.184450 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 638/1260] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:58:51 2022] GET /identity/v3/role_assignments?user.id=dd5b2ba9e37041c78a99f1700a4970c4&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=c60a99de086f45c6820f7872a6b43feb => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:53.174906 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be13df-77b8-479a-9968-8d677ae5a567 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:53.175548 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be13df-77b8-479a-9968-8d677ae5a567 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:53.175548 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be13df-77b8-479a-9968-8d677ae5a567 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:53.176872 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 623/1261] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:58:53.181945 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3130f-9afa-4db6-9144-292a7282511b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:53.182350 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3130f-9afa-4db6-9144-292a7282511b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:53.182506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3130f-9afa-4db6-9144-292a7282511b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:53.199666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-15f3130f-9afa-4db6-9144-292a7282511b None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:58:53.200795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-15f3130f-9afa-4db6-9144-292a7282511b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:58:53.263862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 639/1262] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:58:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 13:58:53.286174 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be13df-77b8-479a-9968-8d677ae5a567 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:53.296781 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:53.383811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be13df-77b8-479a-9968-8d677ae5a567 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:53.416482 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f1687-d761-40b5-979a-382c64ef4d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:53.417255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f1687-d761-40b5-979a-382c64ef4d81 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:53.417540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f1687-d761-40b5-979a-382c64ef4d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:53.417843 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f1687-d761-40b5-979a-382c64ef4d81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:53.427713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f1687-d761-40b5-979a-382c64ef4d81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:58:53.435300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-077f1687-d761-40b5-979a-382c64ef4d81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:58:53.435300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:53.435300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:53.435300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:53.435300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:53.435300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:53.435300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:53.435300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:53.435300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:53.435300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:53.435300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:53.435300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:53.435300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:58:53.435300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:58:53.435300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:58:53.435300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:58:53.436410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:58:53.436410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:58:53.436410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:58:53.436410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:58:53.436410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:53.436410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:53.436410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:53.436410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:53.436410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:53.436410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:53.436410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:53.436410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:53.436410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:53.436410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:53.436410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:53.436410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:53.437325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:53.437325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:53.437325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:53.437325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:53.437325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:58:53.437325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:58:53.437325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:58:53.437325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:58:53.437325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:58:53.437325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:58:53.437325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:58:53.437325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:58:53.437325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 624/1263] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:58:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:53.442394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3130f-9afa-4db6-9144-292a7282511b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:53.462565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:53.540830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3130f-9afa-4db6-9144-292a7282511b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:53.573791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f56853-3dff-4ed6-bd3c-e2ffee101f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:53.575069 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f56853-3dff-4ed6-bd3c-e2ffee101f65 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:53.575358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f56853-3dff-4ed6-bd3c-e2ffee101f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:53.575642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f56853-3dff-4ed6-bd3c-e2ffee101f65 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:53.576416 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f56853-3dff-4ed6-bd3c-e2ffee101f65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:53.576730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f56853-3dff-4ed6-bd3c-e2ffee101f65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:53.578110 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f56853-3dff-4ed6-bd3c-e2ffee101f65 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:53.585887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 640/1264] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:58:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:58:53.591504 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f1687-d761-40b5-979a-382c64ef4d81 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:53.658724 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:53.694959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f1687-d761-40b5-979a-382c64ef4d81 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:53.734258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127e554-6733-49d4-b8c5-c019f50ce8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:53.734978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127e554-6733-49d4-b8c5-c019f50ce8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:53.735261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127e554-6733-49d4-b8c5-c019f50ce8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:53.735538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127e554-6733-49d4-b8c5-c019f50ce8c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:53.746545 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127e554-6733-49d4-b8c5-c019f50ce8c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:53.748142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127e554-6733-49d4-b8c5-c019f50ce8c9 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:53.765275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-2127e554-6733-49d4-b8c5-c019f50ce8c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:53.765275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:53.765275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:53.765275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:53.765275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:53.765275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:53.765275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:53.765275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:53.765275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:53.765275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:53.765275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:53.765275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:53.765275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:58:53.765275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:58:53.765275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:58:53.766328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:58:53.766328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:53.766328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:53.766328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:53.766328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:53.766328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:53.766328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:53.766328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:53.766328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:53.766328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:53.766328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:53.766328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:53.766328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:53.766328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:53.766328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:53.766328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:53.766328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:53.767546 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:58:53.767546 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:58:53.767546 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:53.767546 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:58:53.767546 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 625/1265] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:58:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:53.772212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f56853-3dff-4ed6-bd3c-e2ffee101f65 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:53.850703 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:53.869224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f56853-3dff-4ed6-bd3c-e2ffee101f65 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:53.901363 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222cd2d-c43d-4edc-9c01-8bea3c143263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:53.902016 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222cd2d-c43d-4edc-9c01-8bea3c143263 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:53.902329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222cd2d-c43d-4edc-9c01-8bea3c143263 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:53.902609 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222cd2d-c43d-4edc-9c01-8bea3c143263 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:53.903327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b222cd2d-c43d-4edc-9c01-8bea3c143263 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:53.903639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b222cd2d-c43d-4edc-9c01-8bea3c143263 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:53.905035 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b222cd2d-c43d-4edc-9c01-8bea3c143263 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:53.916286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 641/1266] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:53.922479 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127e554-6733-49d4-b8c5-c019f50ce8c9 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:54.020002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127e554-6733-49d4-b8c5-c019f50ce8c9 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:54.038776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:54.053172 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181cb406-09a2-488e-8b58-3cc59c51572d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:54.053969 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181cb406-09a2-488e-8b58-3cc59c51572d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:54.054259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181cb406-09a2-488e-8b58-3cc59c51572d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:54.054539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181cb406-09a2-488e-8b58-3cc59c51572d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:54.062376 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181cb406-09a2-488e-8b58-3cc59c51572d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:54.063813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181cb406-09a2-488e-8b58-3cc59c51572d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:54.078415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-181cb406-09a2-488e-8b58-3cc59c51572d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 13:58:54.078415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:54.078415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:54.078415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:54.078415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:54.078415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:54.078415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:54.078415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:54.078415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:54.078415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:54.078415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:54.078415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:54.078415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:58:54.078415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:58:54.078415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:58:54.078415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:58:54.079550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:54.079550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:54.079550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:58:54.079550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:58:54.079550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:58:54.079550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:58:54.079550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:54.079550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:54.079550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:54.079550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:54.079550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:54.079550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:54.079550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:54.079550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:54.079550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:54.079550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:54.079550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:54.079550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:54.080601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:54.080601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:54.080601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:58:54.080601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:58:54.080601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:58:54.080601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:58:54.080601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:58:54.080601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:58:54.080601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 13:58:54.080601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:58:54.080601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 626/1267] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:58:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:54.085330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222cd2d-c43d-4edc-9c01-8bea3c143263 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:54.180949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222cd2d-c43d-4edc-9c01-8bea3c143263 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:54.213756 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0031537-f1f0-48f3-becf-164c25c1fa8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:54.214418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0031537-f1f0-48f3-becf-164c25c1fa8f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:54.215311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0031537-f1f0-48f3-becf-164c25c1fa8f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:54.215598 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0031537-f1f0-48f3-becf-164c25c1fa8f None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:54.216459 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0031537-f1f0-48f3-becf-164c25c1fa8f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:54.216777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0031537-f1f0-48f3-becf-164c25c1fa8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:54.218203 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0031537-f1f0-48f3-becf-164c25c1fa8f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:54.226577 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:54.235077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 642/1268] 10.222.0.94 () {56 vars in 1143 bytes} [Mon Jul 18 13:58:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:54.240836 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181cb406-09a2-488e-8b58-3cc59c51572d None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:54.337961 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181cb406-09a2-488e-8b58-3cc59c51572d None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:54.370869 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486bf27-1d5b-4a53-8180-a5b4f535b9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:54.371650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486bf27-1d5b-4a53-8180-a5b4f535b9aa None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:54.371969 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486bf27-1d5b-4a53-8180-a5b4f535b9aa None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:54.372255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486bf27-1d5b-4a53-8180-a5b4f535b9aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:54.383589 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486bf27-1d5b-4a53-8180-a5b4f535b9aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:54.385088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486bf27-1d5b-4a53-8180-a5b4f535b9aa None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:54.401700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-0486bf27-1d5b-4a53-8180-a5b4f535b9aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:54.401700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:54.401700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:54.401700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:54.401700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:54.401700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:54.401700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:54.401700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:54.401700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:54.401700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:54.401700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:54.401700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:54.401700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:58:54.401700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:58:54.401700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:58:54.403300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:58:54.403300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:54.403300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:54.403300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:54.403300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:54.403300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:54.403300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:54.403300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:54.403300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:54.403300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:54.403300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:54.403300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:54.403300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:54.403300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:54.403300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:54.403300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:54.403300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:54.404419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:58:54.404419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:58:54.404419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:54.404419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:58:54.404419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 627/1269] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:58:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:54.409281 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0031537-f1f0-48f3-becf-164c25c1fa8f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:54.504534 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0031537-f1f0-48f3-becf-164c25c1fa8f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:54.536880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e65041-18a6-478d-b8f2-fe359c9ff546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:54.537524 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e65041-18a6-478d-b8f2-fe359c9ff546 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:54.537813 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e65041-18a6-478d-b8f2-fe359c9ff546 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:54.538093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e65041-18a6-478d-b8f2-fe359c9ff546 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:54.538814 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e65041-18a6-478d-b8f2-fe359c9ff546 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:54.539123 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e65041-18a6-478d-b8f2-fe359c9ff546 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:54.540513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e65041-18a6-478d-b8f2-fe359c9ff546 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:54.551839 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 643/1270] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:54.557434 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486bf27-1d5b-4a53-8180-a5b4f535b9aa None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:54.588229 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:54.662208 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486bf27-1d5b-4a53-8180-a5b4f535b9aa None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:54.697286 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b13d1-3400-464b-9b44-c6019cbba7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:54.698333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b13d1-3400-464b-9b44-c6019cbba7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:54.698621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b13d1-3400-464b-9b44-c6019cbba7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:54.699386 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b13d1-3400-464b-9b44-c6019cbba7e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:54.705910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b13d1-3400-464b-9b44-c6019cbba7e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:54.707366 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b13d1-3400-464b-9b44-c6019cbba7e2 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:54.721848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-f40b13d1-3400-464b-9b44-c6019cbba7e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:58:54.721848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:54.721848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:54.721848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:54.721848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:54.721848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:54.721848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:54.721848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:54.721848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:54.721848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:54.721848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:54.721848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:54.721848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:58:54.721848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:58:54.721848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:58:54.722909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:58:54.722909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:54.722909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:54.722909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:54.722909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:54.722909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:54.722909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:54.722909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:54.722909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:54.722909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:54.722909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:54.722909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:54.722909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:54.722909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:54.722909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:54.722909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:54.722909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:54.724047 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:58:54.724047 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:58:54.724047 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:58:54.724047 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:58:54.724047 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:58:54.724047 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:58:54.724047 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:58:54.724047 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:58:54.724047 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 628/1271] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:58:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:54.728850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e65041-18a6-478d-b8f2-fe359c9ff546 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:54.786510 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:54.828050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e65041-18a6-478d-b8f2-fe359c9ff546 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:54.862104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd95f5c8-a613-483d-91a0-9ed5e1ddcb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:54.862722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95f5c8-a613-483d-91a0-9ed5e1ddcb29 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:54.863068 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95f5c8-a613-483d-91a0-9ed5e1ddcb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:54.863376 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd95f5c8-a613-483d-91a0-9ed5e1ddcb29 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:54.864260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95f5c8-a613-483d-91a0-9ed5e1ddcb29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:54.864579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95f5c8-a613-483d-91a0-9ed5e1ddcb29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:54.866115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd95f5c8-a613-483d-91a0-9ed5e1ddcb29 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:54.877912 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 644/1272] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:58:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:54.883844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b13d1-3400-464b-9b44-c6019cbba7e2 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:54.973397 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:54.981957 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b13d1-3400-464b-9b44-c6019cbba7e2 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:55.014507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d5516-60a2-4a0e-9c97-c4b4a6bda10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:55.015380 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d5516-60a2-4a0e-9c97-c4b4a6bda10a None admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:55.015664 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d5516-60a2-4a0e-9c97-c4b4a6bda10a None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:55.015958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d5516-60a2-4a0e-9c97-c4b4a6bda10a None admin] PATH_INFO: `/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/dd5b2ba9e37041c78a99f1700a4970c4/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:55.017872 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d5516-60a2-4a0e-9c97-c4b4a6bda10a None admin] RBAC: Authorizing `identity:create_grant(project_id=ec94946acf91486ba3f5b4fe834b8076, user_id=dd5b2ba9e37041c78a99f1700a4970c4, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:55.019293 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d5516-60a2-4a0e-9c97-c4b4a6bda10a None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:55.032511 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 629/1273] 10.222.0.94 () {58 vars in 1439 bytes} [Mon Jul 18 13:58:54 2022] PUT /identity/v3/projects/ec94946acf91486ba3f5b4fe834b8076/users/dd5b2ba9e37041c78a99f1700a4970c4/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 13:58:57.005953 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec87fd-3966-4f6d-b5b5-c74337ff6882 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:57.006470 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec87fd-3966-4f6d-b5b5-c74337ff6882 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:57.006573 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec87fd-3966-4f6d-b5b5-c74337ff6882 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:57.007865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 645/1274] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:58:57.013623 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52287f91-f49c-4fb5-ab60-a5ca86792eab None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:57.014022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52287f91-f49c-4fb5-ab60-a5ca86792eab None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:57.014188 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52287f91-f49c-4fb5-ab60-a5ca86792eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:57.030934 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-52287f91-f49c-4fb5-ab60-a5ca86792eab None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:58:57.032146 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-52287f91-f49c-4fb5-ab60-a5ca86792eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:58:57.096983 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 630/1275] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:58:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:58:57.119365 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec87fd-3966-4f6d-b5b5-c74337ff6882 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:57.218328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec87fd-3966-4f6d-b5b5-c74337ff6882 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:57.235450 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:57.252546 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d8401-2527-4c48-b7a3-e6130616e202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:57.253327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d8401-2527-4c48-b7a3-e6130616e202 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:57.253619 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d8401-2527-4c48-b7a3-e6130616e202 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:57.253899 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d8401-2527-4c48-b7a3-e6130616e202 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:57.264303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d8401-2527-4c48-b7a3-e6130616e202 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 13:58:57.271863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e86d8401-2527-4c48-b7a3-e6130616e202 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:58:57.271863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:57.271863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:57.271863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:57.271863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:57.271863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:57.271863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:57.271863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:57.271863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:57.271863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:57.271863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:57.271863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:57.271863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 13:58:57.271863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 13:58:57.271863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 13:58:57.271863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 13:58:57.272997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 13:58:57.272997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 13:58:57.272997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 13:58:57.272997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 13:58:57.272997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:57.272997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:57.272997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:57.272997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:57.272997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:57.272997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:57.272997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:57.272997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:57.272997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:57.272997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:57.272997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:57.272997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:57.274061 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:57.274061 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:57.274061 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:57.274061 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:57.274061 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 13:58:57.274061 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 13:58:57.274061 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 13:58:57.274061 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 13:58:57.274061 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 13:58:57.274061 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 13:58:57.274061 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 13:58:57.274061 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:58:57.274061 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 646/1276] 10.222.0.94 () {56 vars in 1096 bytes} [Mon Jul 18 13:58:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:57.279822 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52287f91-f49c-4fb5-ab60-a5ca86792eab None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:57.357110 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:57.375583 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52287f91-f49c-4fb5-ab60-a5ca86792eab None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:57.408241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92b70b-9873-4f49-9542-d3b49e9e9899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:57.408897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92b70b-9873-4f49-9542-d3b49e9e9899 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:57.409185 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92b70b-9873-4f49-9542-d3b49e9e9899 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:57.409464 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92b70b-9873-4f49-9542-d3b49e9e9899 None admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:57.410206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92b70b-9873-4f49-9542-d3b49e9e9899 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:57.410524 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92b70b-9873-4f49-9542-d3b49e9e9899 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:57.411887 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92b70b-9873-4f49-9542-d3b49e9e9899 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:57.419685 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 631/1277] 10.222.0.94 () {56 vars in 1099 bytes} [Mon Jul 18 13:58:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:57.425266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d8401-2527-4c48-b7a3-e6130616e202 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:57.523587 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d8401-2527-4c48-b7a3-e6130616e202 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:57.556963 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d19b9-df6f-46d8-8ce4-cd8267f8f93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:57.557734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d19b9-df6f-46d8-8ce4-cd8267f8f93d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:57.558048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d19b9-df6f-46d8-8ce4-cd8267f8f93d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:57.558332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d19b9-df6f-46d8-8ce4-cd8267f8f93d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:57.568635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d19b9-df6f-46d8-8ce4-cd8267f8f93d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:57.570076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d19b9-df6f-46d8-8ce4-cd8267f8f93d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:57.586946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-096d19b9-df6f-46d8-8ce4-cd8267f8f93d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:57.586946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:57.586946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:57.586946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:57.586946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:57.586946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:57.586946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:57.586946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:57.586946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:57.586946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:57.586946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:57.586946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:57.586946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:58:57.586946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:58:57.586946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:58:57.588039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:58:57.588039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:57.588039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:57.588039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:57.588039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:57.588039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:57.588039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:57.588039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:57.588039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:57.588039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:57.588039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:57.588039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:57.588039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:57.588039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:57.588039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:57.588039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:57.588039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:57.589188 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:58:57.589188 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:58:57.589188 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:57.589188 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:58:57.589188 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 647/1278] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:58:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:57.594175 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92b70b-9873-4f49-9542-d3b49e9e9899 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:57.690674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92b70b-9873-4f49-9542-d3b49e9e9899 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:57.723644 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2265a42-f20c-4e78-a8fe-83312148f975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:57.724322 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2265a42-f20c-4e78-a8fe-83312148f975 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:57.724615 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2265a42-f20c-4e78-a8fe-83312148f975 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:57.724895 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2265a42-f20c-4e78-a8fe-83312148f975 None admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:57.725610 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2265a42-f20c-4e78-a8fe-83312148f975 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:57.725920 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2265a42-f20c-4e78-a8fe-83312148f975 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:57.727286 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2265a42-f20c-4e78-a8fe-83312148f975 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:57.739319 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 632/1279] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:57.744679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d19b9-df6f-46d8-8ce4-cd8267f8f93d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:57.774951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:57.843555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d19b9-df6f-46d8-8ce4-cd8267f8f93d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:57.875567 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b8c41-7315-4761-9460-9957fc27fefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:57.876398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b8c41-7315-4761-9460-9957fc27fefd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:57.876689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b8c41-7315-4761-9460-9957fc27fefd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:57.877493 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b8c41-7315-4761-9460-9957fc27fefd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:57.885248 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b8c41-7315-4761-9460-9957fc27fefd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:57.887239 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b8c41-7315-4761-9460-9957fc27fefd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:57.901147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-179b8c41-7315-4761-9460-9957fc27fefd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 13:58:57.901147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:57.901147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:57.901147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:57.901147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:57.901147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:57.901147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:57.901147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:57.901147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:57.901147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:57.901147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:57.901147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:57.901147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 13:58:57.901147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 13:58:57.901147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 13:58:57.901147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 13:58:57.902384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:57.902384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:57.902384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 13:58:57.902384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:58:57.902384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 13:58:57.902384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 13:58:57.902384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:57.902384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:57.902384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:57.902384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:57.902384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:57.902384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:57.902384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:57.902384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:57.902384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:57.902384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:57.902384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:57.902384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:57.903540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:57.903540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:57.903540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 13:58:57.903540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 13:58:57.903540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 13:58:57.903540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 13:58:57.903540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 13:58:57.903540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 13:58:57.903540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 13:58:57.903540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:58:57.903540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 648/1280] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:58:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:57.908145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2265a42-f20c-4e78-a8fe-83312148f975 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:57.911869 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:58.006084 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2265a42-f20c-4e78-a8fe-83312148f975 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:58.039303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86845c21-ed5e-4ddf-b71c-3db8facdb2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:58.039913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86845c21-ed5e-4ddf-b71c-3db8facdb2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:58.040231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86845c21-ed5e-4ddf-b71c-3db8facdb2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:58.040514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86845c21-ed5e-4ddf-b71c-3db8facdb2e9 None admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:58.041350 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86845c21-ed5e-4ddf-b71c-3db8facdb2e9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:58.041668 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86845c21-ed5e-4ddf-b71c-3db8facdb2e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:58.043125 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86845c21-ed5e-4ddf-b71c-3db8facdb2e9 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:58.059924 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 633/1281] 10.222.0.94 () {56 vars in 1143 bytes} [Mon Jul 18 13:58:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:58.065592 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b8c41-7315-4761-9460-9957fc27fefd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:58.154568 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:58.164607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b8c41-7315-4761-9460-9957fc27fefd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:58.197152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68472b-dbc4-46b2-9b41-b808ad6208a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:58.198056 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68472b-dbc4-46b2-9b41-b808ad6208a5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:58.198347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68472b-dbc4-46b2-9b41-b808ad6208a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:58.199299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68472b-dbc4-46b2-9b41-b808ad6208a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:58.265059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68472b-dbc4-46b2-9b41-b808ad6208a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:58.266508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68472b-dbc4-46b2-9b41-b808ad6208a5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:58.350486 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:58.361115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 649/1282] 10.222.0.94 () {58 vars in 1282 bytes} [Mon Jul 18 13:58:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 13:58:58.366833 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86845c21-ed5e-4ddf-b71c-3db8facdb2e9 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:58.426619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:58.464216 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86845c21-ed5e-4ddf-b71c-3db8facdb2e9 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:58.497097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77300341-cf78-4345-8c36-2d5f5ffbff65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:58.497877 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77300341-cf78-4345-8c36-2d5f5ffbff65 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:58.498164 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77300341-cf78-4345-8c36-2d5f5ffbff65 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:58.498442 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77300341-cf78-4345-8c36-2d5f5ffbff65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:58.508106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77300341-cf78-4345-8c36-2d5f5ffbff65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:58.509541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77300341-cf78-4345-8c36-2d5f5ffbff65 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:58.526275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-77300341-cf78-4345-8c36-2d5f5ffbff65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:58.526275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:58.526275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:58.526275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:58.526275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:58.526275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:58.526275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:58.526275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:58.526275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:58.526275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:58.526275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:58.526275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:58.526275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 13:58:58.526275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 13:58:58.526275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 13:58:58.527370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 13:58:58.527370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:58.527370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:58.527370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:58.527370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:58.527370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:58.527370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:58.527370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:58.527370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:58.527370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:58.527370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:58.527370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:58.527370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:58.527370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:58.527370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:58.527370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:58.527370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:58.528744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 13:58:58.528744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 13:58:58.528744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 13:58:58.528744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:58:58.528744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 634/1283] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 13:58:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:58.533017 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68472b-dbc4-46b2-9b41-b808ad6208a5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:58.627913 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68472b-dbc4-46b2-9b41-b808ad6208a5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:58.660163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbcec3-da7f-43f3-9cfb-59b6b82a8cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:58.660817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbcec3-da7f-43f3-9cfb-59b6b82a8cfa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:58.661104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbcec3-da7f-43f3-9cfb-59b6b82a8cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:58.661383 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbcec3-da7f-43f3-9cfb-59b6b82a8cfa None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:58.662106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebbcec3-da7f-43f3-9cfb-59b6b82a8cfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:58.662447 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebbcec3-da7f-43f3-9cfb-59b6b82a8cfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:58.663853 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebbcec3-da7f-43f3-9cfb-59b6b82a8cfa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:58.675219 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 650/1284] 10.222.0.94 () {56 vars in 1109 bytes} [Mon Jul 18 13:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:58.680660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77300341-cf78-4345-8c36-2d5f5ffbff65 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:58.777493 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77300341-cf78-4345-8c36-2d5f5ffbff65 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:58.790807 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:58.816983 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883074d-5d2c-4781-ae79-8fe5b9252d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:58.818318 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883074d-5d2c-4781-ae79-8fe5b9252d61 None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:58.818792 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883074d-5d2c-4781-ae79-8fe5b9252d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:58.819083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883074d-5d2c-4781-ae79-8fe5b9252d61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:58.827743 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883074d-5d2c-4781-ae79-8fe5b9252d61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:58.829228 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5883074d-5d2c-4781-ae79-8fe5b9252d61 None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:58.843028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application [None req-5883074d-5d2c-4781-ae79-8fe5b9252d61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:58:58.843028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:58:58.843028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:58:58.843028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:58:58.843028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:58:58.843028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:58:58.843028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:58:58.843028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:58:58.843028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:58:58.843028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:58:58.843028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:58:58.843028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:58:58.843028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 13:58:58.843028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 13:58:58.843028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 13:58:58.844124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 13:58:58.844124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:58:58.844124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:58:58.844124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:58:58.844124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:58:58.844124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:58:58.844124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:58:58.844124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:58:58.844124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application with Lock( Jul 18 13:58:58.844124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:58:58.844124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:58:58.844124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:58:58.844124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:58:58.844124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:58:58.844124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:58:58.844124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:58:58.844124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:58:58.845423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 13:58:58.845423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 13:58:58.845423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 13:58:58.845423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 13:58:58.845423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 13:58:58.845423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 13:58:58.845423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 13:58:58.845423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: ERROR keystone.server.flask.application Jul 18 13:58:58.845423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 635/1285] 10.222.0.94 () {56 vars in 1111 bytes} [Mon Jul 18 13:58:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:58:58.849994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbcec3-da7f-43f3-9cfb-59b6b82a8cfa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:58.899844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:58:58.949587 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbcec3-da7f-43f3-9cfb-59b6b82a8cfa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:58.982122 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c70c0-f984-434a-b138-5f6912539e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:58.982784 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c70c0-f984-434a-b138-5f6912539e65 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:58.983056 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c70c0-f984-434a-b138-5f6912539e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:58.983336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c70c0-f984-434a-b138-5f6912539e65 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:58.984151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c70c0-f984-434a-b138-5f6912539e65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:58.984464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c70c0-f984-434a-b138-5f6912539e65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:58.985885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c70c0-f984-434a-b138-5f6912539e65 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:58.997898 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 651/1286] 10.222.0.94 () {56 vars in 1148 bytes} [Mon Jul 18 13:58:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:58:59.004089 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883074d-5d2c-4781-ae79-8fe5b9252d61 None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:58:59.103240 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883074d-5d2c-4781-ae79-8fe5b9252d61 None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:58:59.135934 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b37ce0-b720-4288-b429-0db2df64ed4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:58:59.137243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b37ce0-b720-4288-b429-0db2df64ed4d None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:58:59.137510 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b37ce0-b720-4288-b429-0db2df64ed4d None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:58:59.137789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b37ce0-b720-4288-b429-0db2df64ed4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:58:59.138616 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b37ce0-b720-4288-b429-0db2df64ed4d None admin] RBAC: Adding query filter params (role.id=c60a99de086f45c6820f7872a6b43feb, scope.project.id=ec94946acf91486ba3f5b4fe834b8076, user.id=dd5b2ba9e37041c78a99f1700a4970c4) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:58:59.138932 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b37ce0-b720-4288-b429-0db2df64ed4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:58:59.140382 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b37ce0-b720-4288-b429-0db2df64ed4d None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:58:59.146066 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 636/1287] 10.222.0.94 () {56 vars in 1374 bytes} [Mon Jul 18 13:58:59 2022] GET /identity/v3/role_assignments?user.id=dd5b2ba9e37041c78a99f1700a4970c4&scope.project.id=ec94946acf91486ba3f5b4fe834b8076&role.id=c60a99de086f45c6820f7872a6b43feb => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:59:01.115722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c34de2-7c25-4b01-a863-4c4ae42c33b9 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:59:01.116279 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c34de2-7c25-4b01-a863-4c4ae42c33b9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:59:01.116395 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c34de2-7c25-4b01-a863-4c4ae42c33b9 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:59:01.117666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 652/1288] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:59:01.123210 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbbd32-041f-49f3-9dac-de92a35c014e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:59:01.123582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbbd32-041f-49f3-9dac-de92a35c014e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:59:01.123773 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbbd32-041f-49f3-9dac-de92a35c014e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:59:01.141006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d4cbbd32-041f-49f3-9dac-de92a35c014e None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:59:01.142175 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d4cbbd32-041f-49f3-9dac-de92a35c014e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:59:01.170218 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:59:01.208074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 637/1289] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:59:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:59:01.230393 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c34de2-7c25-4b01-a863-4c4ae42c33b9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:59:01.330054 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c34de2-7c25-4b01-a863-4c4ae42c33b9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:59:01.341010 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:59:01.364198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e3edf-e59a-4f0c-8907-df2c43ffca6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:59:01.365020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e3edf-e59a-4f0c-8907-df2c43ffca6c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:59:01.365331 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e3edf-e59a-4f0c-8907-df2c43ffca6c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:59:01.365615 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e3edf-e59a-4f0c-8907-df2c43ffca6c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:59:01.366273 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e3edf-e59a-4f0c-8907-df2c43ffca6c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:59:01.367647 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e3edf-e59a-4f0c-8907-df2c43ffca6c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:59:01.379241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b79e3edf-e59a-4f0c-8907-df2c43ffca6c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 13:59:01.379241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:59:01.379241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:59:01.379241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:59:01.379241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:59:01.379241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:59:01.379241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:59:01.379241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:59:01.379241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 13:59:01.379241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:59:01.379241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:59:01.379241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:59:01.379241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:59:01.379241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:59:01.379241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:59:01.379241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:59:01.379241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:59:01.380513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:59:01.380513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:59:01.380513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:59:01.380513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:59:01.380513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:59:01.380513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:59:01.380513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:59:01.380513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:59:01.380513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:59:01.380513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:59:01.380513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:59:01.380513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:59:01.380513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:59:01.380513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:59:01.380513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:59:01.380513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:59:01.381484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:59:01.381484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:59:01.381484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:59:01.381484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:59:01.381484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:59:01.381484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:59:01.381484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:59:01.381484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:59:01.381484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:59:01.381484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:59:01.381484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:59:01.381484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:59:01.381484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 13:59:01.381484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:59:01.381484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 653/1290] 10.222.0.94 () {56 vars in 1117 bytes} [Mon Jul 18 13:59:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:59:01.386914 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbbd32-041f-49f3-9dac-de92a35c014e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:59:01.482049 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbbd32-041f-49f3-9dac-de92a35c014e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:59:01.514044 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd86e2e-93ca-4b1e-a4ab-4582834dfd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:59:01.514701 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd86e2e-93ca-4b1e-a4ab-4582834dfd0b None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:59:01.514986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd86e2e-93ca-4b1e-a4ab-4582834dfd0b None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:59:01.515264 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd86e2e-93ca-4b1e-a4ab-4582834dfd0b None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:59:01.515978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd86e2e-93ca-4b1e-a4ab-4582834dfd0b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:59:01.516288 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd86e2e-93ca-4b1e-a4ab-4582834dfd0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:59:01.517656 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd86e2e-93ca-4b1e-a4ab-4582834dfd0b None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:59:01.522322 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 638/1291] 10.222.0.94 () {56 vars in 1116 bytes} [Mon Jul 18 13:59:01 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:59:01.529802 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e3edf-e59a-4f0c-8907-df2c43ffca6c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:59:01.624766 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e3edf-e59a-4f0c-8907-df2c43ffca6c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:59:01.660824 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93713045-c839-45e8-bd38-7c5d65e7a8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:59:01.661527 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93713045-c839-45e8-bd38-7c5d65e7a8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:59:01.661806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93713045-c839-45e8-bd38-7c5d65e7a8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:59:01.662083 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93713045-c839-45e8-bd38-7c5d65e7a8e1 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:59:01.662784 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93713045-c839-45e8-bd38-7c5d65e7a8e1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:59:01.663096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93713045-c839-45e8-bd38-7c5d65e7a8e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:59:01.664495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93713045-c839-45e8-bd38-7c5d65e7a8e1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:59:01.669185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 654/1292] 10.222.0.94 () {56 vars in 1116 bytes} [Mon Jul 18 13:59:01 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:59:03.601304 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c129daf-7634-44fd-88f5-a1dbd523ec5f None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:59:03.601924 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c129daf-7634-44fd-88f5-a1dbd523ec5f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:59:03.601924 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c129daf-7634-44fd-88f5-a1dbd523ec5f None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:59:03.603214 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 639/1293] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:59:03.608842 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a41b28-c8c5-4013-b39d-2f9f2cdee724 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:59:03.609268 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a41b28-c8c5-4013-b39d-2f9f2cdee724 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:59:03.609414 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a41b28-c8c5-4013-b39d-2f9f2cdee724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:59:03.625841 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-89a41b28-c8c5-4013-b39d-2f9f2cdee724 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:59:03.627035 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-89a41b28-c8c5-4013-b39d-2f9f2cdee724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:59:03.656619 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:59:03.693451 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 655/1294] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:59:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:59:03.716584 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c129daf-7634-44fd-88f5-a1dbd523ec5f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:59:03.746721 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:59:03.814979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c129daf-7634-44fd-88f5-a1dbd523ec5f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:59:03.847765 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b023d-a813-4381-a08e-804888fe1a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:59:03.848467 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b023d-a813-4381-a08e-804888fe1a4c None admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:59:03.848759 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b023d-a813-4381-a08e-804888fe1a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:59:03.849047 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b023d-a813-4381-a08e-804888fe1a4c None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:59:03.849780 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b023d-a813-4381-a08e-804888fe1a4c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:59:03.851143 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b023d-a813-4381-a08e-804888fe1a4c None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:59:03.863866 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 640/1295] 10.222.0.94 () {60 vars in 1141 bytes} [Mon Jul 18 13:59:03 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:59:05.872181 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b57d38-b7c6-4eff-9dc5-3e39145f5307 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:59:05.872735 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b57d38-b7c6-4eff-9dc5-3e39145f5307 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:59:05.872735 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b57d38-b7c6-4eff-9dc5-3e39145f5307 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:59:05.874098 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 656/1296] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:59:05.879235 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6182be-dd73-49c0-994a-b7a5901c1266 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:59:05.879653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6182be-dd73-49c0-994a-b7a5901c1266 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:59:05.879802 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6182be-dd73-49c0-994a-b7a5901c1266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:59:05.896863 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ae6182be-dd73-49c0-994a-b7a5901c1266 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:59:05.898048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ae6182be-dd73-49c0-994a-b7a5901c1266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:59:05.969613 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 641/1297] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:59:05 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:59:05.992064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b57d38-b7c6-4eff-9dc5-3e39145f5307 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:59:06.089923 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b57d38-b7c6-4eff-9dc5-3e39145f5307 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:59:06.123410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c0a9a-537d-455c-a5e2-624b1dee0157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:59:06.124169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c0a9a-537d-455c-a5e2-624b1dee0157 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:59:06.124459 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c0a9a-537d-455c-a5e2-624b1dee0157 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:59:06.124738 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c0a9a-537d-455c-a5e2-624b1dee0157 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:59:06.125396 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c0a9a-537d-455c-a5e2-624b1dee0157 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:59:06.126777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c0a9a-537d-455c-a5e2-624b1dee0157 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:59:06.139825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-918c0a9a-537d-455c-a5e2-624b1dee0157 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 13:59:06.139825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:59:06.139825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:59:06.139825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:59:06.139825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:59:06.139825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:59:06.139825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:59:06.139825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:59:06.139825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 13:59:06.139825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:59:06.139825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:59:06.139825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:59:06.139825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:59:06.139825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:59:06.139825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:59:06.139825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:59:06.139825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:59:06.141169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:59:06.141169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:59:06.141169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:59:06.141169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:59:06.141169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:59:06.141169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:59:06.141169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:59:06.141169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:59:06.141169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:59:06.141169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:59:06.141169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:59:06.141169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:59:06.141169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:59:06.141169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:59:06.141169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:59:06.141169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:59:06.142327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:59:06.142327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:59:06.142327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:59:06.142327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:59:06.142327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:59:06.142327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:59:06.142327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:59:06.142327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:59:06.142327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:59:06.142327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:59:06.142327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:59:06.142327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:59:06.142327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 13:59:06.142327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:59:06.142327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 657/1298] 10.222.0.94 () {56 vars in 1117 bytes} [Mon Jul 18 13:59:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:59:06.147296 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6182be-dd73-49c0-994a-b7a5901c1266 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:59:06.165687 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:59:06.243949 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6182be-dd73-49c0-994a-b7a5901c1266 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:59:06.277043 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d04c1-8f46-4201-acfb-f6f40fdf21bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:59:06.277698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d04c1-8f46-4201-acfb-f6f40fdf21bb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:59:06.277984 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d04c1-8f46-4201-acfb-f6f40fdf21bb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:59:06.278262 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d04c1-8f46-4201-acfb-f6f40fdf21bb None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:59:06.278960 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d04c1-8f46-4201-acfb-f6f40fdf21bb None admin] RBAC: Adding query filter params (name=placement) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:59:06.279269 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d04c1-8f46-4201-acfb-f6f40fdf21bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:59:06.280657 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d04c1-8f46-4201-acfb-f6f40fdf21bb None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:59:06.285660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 642/1299] 10.222.0.94 () {56 vars in 1116 bytes} [Mon Jul 18 13:59:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 13:59:06.291424 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c0a9a-537d-455c-a5e2-624b1dee0157 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:59:06.322611 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:59:06.389503 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c0a9a-537d-455c-a5e2-624b1dee0157 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:59:06.422331 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19a65d-1d19-49d0-b56e-888bf474651f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:59:06.422971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19a65d-1d19-49d0-b56e-888bf474651f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:59:06.423260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19a65d-1d19-49d0-b56e-888bf474651f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:59:06.423539 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19a65d-1d19-49d0-b56e-888bf474651f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:59:06.424360 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19a65d-1d19-49d0-b56e-888bf474651f None admin] RBAC: Adding query filter params (interface=public, service_id=531e7e882e2743419fcee7dd02d535c2, region_id=RegionOne) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:59:06.424676 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19a65d-1d19-49d0-b56e-888bf474651f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:59:06.426036 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19a65d-1d19-49d0-b56e-888bf474651f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:59:06.430892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 658/1300] 10.222.0.94 () {56 vars in 1251 bytes} [Mon Jul 18 13:59:06 2022] GET /identity/v3/endpoints?service_id=531e7e882e2743419fcee7dd02d535c2&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:59:06.436882 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d04c1-8f46-4201-acfb-f6f40fdf21bb None admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:59:06.515595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:59:06.533167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d04c1-8f46-4201-acfb-f6f40fdf21bb None admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:59:06.565593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbd3b9-f08c-4e03-baa4-a62f37657ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:59:06.566243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbd3b9-f08c-4e03-baa4-a62f37657ecc None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:59:06.566530 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbd3b9-f08c-4e03-baa4-a62f37657ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:59:06.566808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbd3b9-f08c-4e03-baa4-a62f37657ecc None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:59:06.567482 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbd3b9-f08c-4e03-baa4-a62f37657ecc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:59:06.568864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbd3b9-f08c-4e03-baa4-a62f37657ecc None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:59:06.576370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 643/1301] 10.222.0.94 () {56 vars in 1087 bytes} [Mon Jul 18 13:59:06 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 13:59:08.587323 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86ad71-86f1-4d71-980b-692a74aa1b62 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:59:08.587889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86ad71-86f1-4d71-980b-692a74aa1b62 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:59:08.587889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86ad71-86f1-4d71-980b-692a74aa1b62 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:59:08.589753 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 659/1302] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 13:59:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 13:59:08.594796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b56443-b90a-4fe4-9a66-966dc60efca0 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:59:08.595191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b56443-b90a-4fe4-9a66-966dc60efca0 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:59:08.595373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b56443-b90a-4fe4-9a66-966dc60efca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:59:08.612291 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-93b56443-b90a-4fe4-9a66-966dc60efca0 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 13:59:08.613477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-93b56443-b90a-4fe4-9a66-966dc60efca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 13:59:08.682572 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 644/1303] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 13:59:08 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 13:59:08.708198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86ad71-86f1-4d71-980b-692a74aa1b62 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:59:08.805031 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:59:08.812654 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86ad71-86f1-4d71-980b-692a74aa1b62 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:59:08.845177 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937e8f8-e862-4e01-abcc-c13b492551da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:59:08.845907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937e8f8-e862-4e01-abcc-c13b492551da None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:59:08.846198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937e8f8-e862-4e01-abcc-c13b492551da None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:59:08.846477 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937e8f8-e862-4e01-abcc-c13b492551da None admin] PATH_INFO: `/v3/services/placement` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:59:08.847123 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937e8f8-e862-4e01-abcc-c13b492551da None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:59:08.848516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937e8f8-e862-4e01-abcc-c13b492551da None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:59:08.860076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-9937e8f8-e862-4e01-abcc-c13b492551da None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 13:59:08.860076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:59:08.860076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 13:59:08.860076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 13:59:08.860076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 13:59:08.860076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 13:59:08.860076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 13:59:08.860076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:59:08.860076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 13:59:08.860076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:59:08.860076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 13:59:08.860076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:59:08.860076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 13:59:08.860076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 13:59:08.860076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 13:59:08.860076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 13:59:08.860076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 13:59:08.861229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 13:59:08.861229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 13:59:08.861229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 13:59:08.861229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 13:59:08.861229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 13:59:08.861229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 13:59:08.861229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 13:59:08.861229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 13:59:08.861229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 13:59:08.861229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 13:59:08.861229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 13:59:08.861229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 13:59:08.861229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 13:59:08.861229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 13:59:08.861229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 13:59:08.861229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 13:59:08.862537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 13:59:08.862537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jul 18 13:59:08.862537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 13:59:08.862537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jul 18 13:59:08.862537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 13:59:08.862537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 13:59:08.862537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 13:59:08.862537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jul 18 13:59:08.862537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 13:59:08.862537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jul 18 13:59:08.862537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 13:59:08.862537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 13:59:08.862537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 13:59:08.862537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jul 18 13:59:08.862537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 660/1304] 10.222.0.94 () {56 vars in 1117 bytes} [Mon Jul 18 13:59:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 13:59:08.867299 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b56443-b90a-4fe4-9a66-966dc60efca0 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:59:08.941865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:59:08.964436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b56443-b90a-4fe4-9a66-966dc60efca0 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:59:08.997358 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fd358-7d0b-4401-9e52-097f804243cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:59:08.998012 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fd358-7d0b-4401-9e52-097f804243cb None admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:59:08.998295 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fd358-7d0b-4401-9e52-097f804243cb None admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:59:08.998569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fd358-7d0b-4401-9e52-097f804243cb None admin] PATH_INFO: `/v3/services` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:59:08.999269 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2fd358-7d0b-4401-9e52-097f804243cb None admin] RBAC: Adding query filter params (name=placement) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 13:59:08.999575 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2fd358-7d0b-4401-9e52-097f804243cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:59:09.000956 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2fd358-7d0b-4401-9e52-097f804243cb None admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:59:09.005948 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 645/1305] 10.222.0.94 () {56 vars in 1116 bytes} [Mon Jul 18 13:59:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 13:59:09.011758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937e8f8-e862-4e01-abcc-c13b492551da None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 13:59:09.108026 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937e8f8-e862-4e01-abcc-c13b492551da None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 13:59:09.139701 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8dd725-cfd6-4e53-8fa9-ef5900a51ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 13:59:09.140379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8dd725-cfd6-4e53-8fa9-ef5900a51ed2 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 13:59:09.140664 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8dd725-cfd6-4e53-8fa9-ef5900a51ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 13:59:09.140940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8dd725-cfd6-4e53-8fa9-ef5900a51ed2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 13:59:09.141667 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8dd725-cfd6-4e53-8fa9-ef5900a51ed2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 13:59:09.143019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8dd725-cfd6-4e53-8fa9-ef5900a51ed2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 13:59:09.160060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 661/1306] 10.222.0.94 () {60 vars in 1144 bytes} [Mon Jul 18 13:59:09 2022] POST /identity/v3/endpoints => generated 331 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 13:59:19.002953 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=30 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 13:59:19.155903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:11.312615 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f950c-89a3-4582-9371-56fc586531a4 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:11.313660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f950c-89a3-4582-9371-56fc586531a4 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:11.314153 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f950c-89a3-4582-9371-56fc586531a4 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:11.321034 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 646/1307] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:00:11 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:00:11.322512 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a8319-7ce9-4e14-a619-acdb34bb3a66 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:11.322971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a8319-7ce9-4e14-a619-acdb34bb3a66 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:11.323065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a8319-7ce9-4e14-a619-acdb34bb3a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:11.415108 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b61a8319-7ce9-4e14-a619-acdb34bb3a66 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:00:11.416285 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b61a8319-7ce9-4e14-a619-acdb34bb3a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:11.531831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 662/1308] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:00:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:00:11.581856 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbb8ad-f133-4e3a-8883-a6cabb5dad73 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:11.582403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbb8ad-f133-4e3a-8883-a6cabb5dad73 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:11.583362 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbb8ad-f133-4e3a-8883-a6cabb5dad73 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:11.585388 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 647/1309] 10.222.0.94 () {54 vars in 965 bytes} [Mon Jul 18 14:00:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:00:11.592477 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f3809-ab87-40f5-aedb-bf6c527bb2d9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:11.592929 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f3809-ab87-40f5-aedb-bf6c527bb2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:11.593318 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f3809-ab87-40f5-aedb-bf6c527bb2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:11.747880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-142f3809-ab87-40f5-aedb-bf6c527bb2d9 None None] MFA Rules not processed for user `14027a0e13fd4272bd7803e3131b6348`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:00:11.750629 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-142f3809-ab87-40f5-aedb-bf6c527bb2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:11.798840 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:11.895070 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 663/1310] 10.222.0.94 () {58 vars in 1062 bytes} [Mon Jul 18 14:00:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:00:11.903704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbb8ad-f133-4e3a-8883-a6cabb5dad73 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:00:11.905466 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-28bbb8ad-f133-4e3a-8883-a6cabb5dad73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:12.018757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbb8ad-f133-4e3a-8883-a6cabb5dad73 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:00:12.096021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5301c5-1fd9-4316-a248-82e0cf2596c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:00:12.096587 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5301c5-1fd9-4316-a248-82e0cf2596c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:12.096873 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5301c5-1fd9-4316-a248-82e0cf2596c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:12.097151 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5301c5-1fd9-4316-a248-82e0cf2596c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:12.098808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9d5301c5-1fd9-4316-a248-82e0cf2596c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:12.154751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:12.216069 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5301c5-1fd9-4316-a248-82e0cf2596c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:00:12.217510 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5301c5-1fd9-4316-a248-82e0cf2596c5 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:00:12.236421 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 648/1311] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:00:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:00:14.498307 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59007c-9cdc-457f-9aa7-cdd3154e9b71 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:14.498868 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59007c-9cdc-457f-9aa7-cdd3154e9b71 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:14.498868 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59007c-9cdc-457f-9aa7-cdd3154e9b71 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:14.500312 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 664/1312] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:00:14.506206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae1ca9-2eea-4a97-8ae3-e6cf8e817003 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:14.506657 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae1ca9-2eea-4a97-8ae3-e6cf8e817003 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:14.506750 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae1ca9-2eea-4a97-8ae3-e6cf8e817003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:14.575875 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c7ae1ca9-2eea-4a97-8ae3-e6cf8e817003 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:00:14.577088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c7ae1ca9-2eea-4a97-8ae3-e6cf8e817003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:14.680461 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 649/1313] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:00:14.719788 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59007c-9cdc-457f-9aa7-cdd3154e9b71 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:00:14.721394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1c59007c-9cdc-457f-9aa7-cdd3154e9b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:14.845932 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59007c-9cdc-457f-9aa7-cdd3154e9b71 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:00:14.927254 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:14.929272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7029e-e350-4e93-881e-28064c7ace7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:00:14.929945 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7029e-e350-4e93-881e-28064c7ace7b service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:14.930319 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7029e-e350-4e93-881e-28064c7ace7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:14.930601 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7029e-e350-4e93-881e-28064c7ace7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:14.933289 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-85f7029e-e350-4e93-881e-28064c7ace7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:15.054254 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7029e-e350-4e93-881e-28064c7ace7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:00:15.055818 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7029e-e350-4e93-881e-28064c7ace7b service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:00:15.075043 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 665/1314] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:00:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:00:17.406350 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011d165-e81e-495d-bc5c-086070ba3f93 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:17.406861 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011d165-e81e-495d-bc5c-086070ba3f93 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:17.406861 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011d165-e81e-495d-bc5c-086070ba3f93 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:17.408268 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 650/1315] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:00:17.414027 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263a078-cfc8-402c-8e01-7fec8dda6781 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:17.414472 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263a078-cfc8-402c-8e01-7fec8dda6781 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:17.414591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263a078-cfc8-402c-8e01-7fec8dda6781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:17.481191 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5263a078-cfc8-402c-8e01-7fec8dda6781 None None] MFA Rules not processed for user `2ecf077c36454bf782f946577207b833`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:00:17.482417 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5263a078-cfc8-402c-8e01-7fec8dda6781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:17.533740 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:17.579233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 666/1316] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:00:17 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:00:17.618558 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011d165-e81e-495d-bc5c-086070ba3f93 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:00:17.620094 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a011d165-e81e-495d-bc5c-086070ba3f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:17.630202 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:17.720648 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011d165-e81e-495d-bc5c-086070ba3f93 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:00:17.790527 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4377f-66b6-4537-a3be-6b002a1b0229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:00:17.791166 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4377f-66b6-4537-a3be-6b002a1b0229 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:17.791451 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4377f-66b6-4537-a3be-6b002a1b0229 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:17.791735 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4377f-66b6-4537-a3be-6b002a1b0229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:17.793373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ffa4377f-66b6-4537-a3be-6b002a1b0229 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:17.860820 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:17.909520 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4377f-66b6-4537-a3be-6b002a1b0229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:00:17.910975 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4377f-66b6-4537-a3be-6b002a1b0229 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:00:17.929170 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 651/1317] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:00:17 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:00:20.479329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b367f-dd7c-4ffc-88c6-ac17dcd5670e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:20.479858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b367f-dd7c-4ffc-88c6-ac17dcd5670e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:20.479858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b367f-dd7c-4ffc-88c6-ac17dcd5670e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:20.481275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 667/1318] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:00:20.486349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf5b69-8104-48fe-bf78-d776b014e7b1 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:20.486773 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf5b69-8104-48fe-bf78-d776b014e7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:20.486918 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf5b69-8104-48fe-bf78-d776b014e7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:20.549660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-eccf5b69-8104-48fe-bf78-d776b014e7b1 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:00:20.551382 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-eccf5b69-8104-48fe-bf78-d776b014e7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:20.646860 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 652/1319] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:00:20 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:00:20.685215 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b367f-dd7c-4ffc-88c6-ac17dcd5670e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:00:20.686736 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-116b367f-dd7c-4ffc-88c6-ac17dcd5670e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:20.786191 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b367f-dd7c-4ffc-88c6-ac17dcd5670e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:00:20.855869 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b7ef1-2704-4753-ab7b-dd8941ed9e1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:00:20.856529 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b7ef1-2704-4753-ab7b-dd8941ed9e1f service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:20.856813 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b7ef1-2704-4753-ab7b-dd8941ed9e1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:20.857090 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b7ef1-2704-4753-ab7b-dd8941ed9e1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:20.858692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e61b7ef1-2704-4753-ab7b-dd8941ed9e1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:20.866369 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:20.965337 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b7ef1-2704-4753-ab7b-dd8941ed9e1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:00:20.966798 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b7ef1-2704-4753-ab7b-dd8941ed9e1f service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:00:20.986392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 668/1320] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:00:20 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:00:23.661535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae56123-fb10-4327-a87d-89d45790e9f0 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:23.662070 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae56123-fb10-4327-a87d-89d45790e9f0 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:23.662185 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae56123-fb10-4327-a87d-89d45790e9f0 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:23.663472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 653/1321] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:00:23.668600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a7b19-04f9-43ca-b80f-775e2f14e57d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:23.669065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a7b19-04f9-43ca-b80f-775e2f14e57d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:23.669287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a7b19-04f9-43ca-b80f-775e2f14e57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:23.735659 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-653a7b19-04f9-43ca-b80f-775e2f14e57d None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:00:23.736922 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-653a7b19-04f9-43ca-b80f-775e2f14e57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:23.793980 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:23.838732 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 669/1322] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:00:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:00:23.875405 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae56123-fb10-4327-a87d-89d45790e9f0 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:00:23.877007 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bae56123-fb10-4327-a87d-89d45790e9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:23.898437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:23.990063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae56123-fb10-4327-a87d-89d45790e9f0 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:00:24.063624 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263eb8c9-61a9-4e21-b0f4-ce11ad8e6ced service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:00:24.064307 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263eb8c9-61a9-4e21-b0f4-ce11ad8e6ced service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:24.064597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263eb8c9-61a9-4e21-b0f4-ce11ad8e6ced service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:24.064875 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263eb8c9-61a9-4e21-b0f4-ce11ad8e6ced service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:24.067026 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-263eb8c9-61a9-4e21-b0f4-ce11ad8e6ced service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:24.136751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:24.175008 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263eb8c9-61a9-4e21-b0f4-ce11ad8e6ced service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:00:24.176499 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263eb8c9-61a9-4e21-b0f4-ce11ad8e6ced service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:00:24.197122 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 654/1323] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:00:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:00:26.286398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcd6d3-8ff2-4ebc-8b14-452e2665126d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:26.286909 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcd6d3-8ff2-4ebc-8b14-452e2665126d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:26.286909 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcd6d3-8ff2-4ebc-8b14-452e2665126d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:26.288263 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 670/1324] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:00:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:00:26.293530 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c74aa4-9de0-4694-a05f-338d704d2620 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:26.293942 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c74aa4-9de0-4694-a05f-338d704d2620 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:26.294308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c74aa4-9de0-4694-a05f-338d704d2620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:26.360438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-26c74aa4-9de0-4694-a05f-338d704d2620 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:00:26.361666 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-26c74aa4-9de0-4694-a05f-338d704d2620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:26.461262 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 655/1325] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:00:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:00:26.498418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fcd6d3-8ff2-4ebc-8b14-452e2665126d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:00:26.500001 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a6fcd6d3-8ff2-4ebc-8b14-452e2665126d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:26.602613 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fcd6d3-8ff2-4ebc-8b14-452e2665126d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:00:26.677220 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d830755-e665-4094-98c8-76bc63df7adf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:00:26.677878 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d830755-e665-4094-98c8-76bc63df7adf service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:26.678168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d830755-e665-4094-98c8-76bc63df7adf service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:26.678455 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d830755-e665-4094-98c8-76bc63df7adf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:26.680134 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1d830755-e665-4094-98c8-76bc63df7adf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:26.699127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:26.794928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d830755-e665-4094-98c8-76bc63df7adf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:00:26.796408 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d830755-e665-4094-98c8-76bc63df7adf service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:00:26.813898 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 671/1326] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:00:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:00:28.851248 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f1cc3-896d-422a-9d72-a929cb543f14 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:28.851772 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f1cc3-896d-422a-9d72-a929cb543f14 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:28.851772 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f1cc3-896d-422a-9d72-a929cb543f14 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:28.853189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 656/1327] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:00:28.858267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de876f01-3319-4638-8916-574b660b5a06 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:28.858695 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de876f01-3319-4638-8916-574b660b5a06 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:28.858845 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de876f01-3319-4638-8916-574b660b5a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:28.923777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-de876f01-3319-4638-8916-574b660b5a06 None None] MFA Rules not processed for user `2ecf077c36454bf782f946577207b833`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:00:28.925031 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-de876f01-3319-4638-8916-574b660b5a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:28.990836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:29.023836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 672/1328] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:00:28 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:00:29.062032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f1cc3-896d-422a-9d72-a929cb543f14 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:00:29.063586 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3e0f1cc3-896d-422a-9d72-a929cb543f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:29.098002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:29.169625 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f1cc3-896d-422a-9d72-a929cb543f14 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:00:29.246064 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc2cb5-0950-4f20-8269-ad60a9b3ba12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:00:29.246628 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc2cb5-0950-4f20-8269-ad60a9b3ba12 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:29.246912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc2cb5-0950-4f20-8269-ad60a9b3ba12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:29.247187 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc2cb5-0950-4f20-8269-ad60a9b3ba12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:29.248831 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-cfbc2cb5-0950-4f20-8269-ad60a9b3ba12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:29.326346 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:29.358196 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc2cb5-0950-4f20-8269-ad60a9b3ba12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:00:29.359637 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc2cb5-0950-4f20-8269-ad60a9b3ba12 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:00:29.377752 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 657/1329] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:00:29 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:00:31.622617 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b887082-c0f0-4a85-a380-03a34aff9b57 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:31.623144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b887082-c0f0-4a85-a380-03a34aff9b57 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:31.623144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b887082-c0f0-4a85-a380-03a34aff9b57 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:31.624523 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 673/1330] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:00:31.630176 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f8e1c-ae2b-4c99-9737-68248f4179f1 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:31.630550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f8e1c-ae2b-4c99-9737-68248f4179f1 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:31.630735 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f8e1c-ae2b-4c99-9737-68248f4179f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:31.694541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-741f8e1c-ae2b-4c99-9737-68248f4179f1 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:00:31.695728 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-741f8e1c-ae2b-4c99-9737-68248f4179f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:31.790520 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 658/1331] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:00:31.826504 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b887082-c0f0-4a85-a380-03a34aff9b57 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:00:31.828066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7b887082-c0f0-4a85-a380-03a34aff9b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:31.930432 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b887082-c0f0-4a85-a380-03a34aff9b57 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:00:32.002234 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166782a-7fb4-495d-8f24-c62dfd469622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:00:32.002909 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166782a-7fb4-495d-8f24-c62dfd469622 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:32.003197 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166782a-7fb4-495d-8f24-c62dfd469622 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:32.003478 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166782a-7fb4-495d-8f24-c62dfd469622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:32.005147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4166782a-7fb4-495d-8f24-c62dfd469622 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:32.031690 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:32.112810 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166782a-7fb4-495d-8f24-c62dfd469622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:00:32.114247 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166782a-7fb4-495d-8f24-c62dfd469622 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:00:32.132259 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 674/1332] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:00:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:00:34.189670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a3c0b-d379-4606-8085-58cca1f36172 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:34.190837 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a3c0b-d379-4606-8085-58cca1f36172 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:34.190837 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a3c0b-d379-4606-8085-58cca1f36172 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:34.192467 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 659/1333] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:00:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:00:34.197759 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b2d0e-7ceb-4bd7-8942-f7b5348a2277 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:34.198164 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b2d0e-7ceb-4bd7-8942-f7b5348a2277 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:34.198271 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b2d0e-7ceb-4bd7-8942-f7b5348a2277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:34.264658 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-554b2d0e-7ceb-4bd7-8942-f7b5348a2277 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:00:34.266163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-554b2d0e-7ceb-4bd7-8942-f7b5348a2277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:34.340792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:34.365980 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 675/1334] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:00:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:00:34.421500 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ab3b6-8cb7-4642-802c-6506847e03d9 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:34.421820 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ab3b6-8cb7-4642-802c-6506847e03d9 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:34.422104 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ab3b6-8cb7-4642-802c-6506847e03d9 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:34.423489 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 660/1335] 10.222.0.94 () {54 vars in 965 bytes} [Mon Jul 18 14:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:00:34.431047 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d4805-7963-4a3b-8dc8-9de72ef96ba9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:34.431343 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d4805-7963-4a3b-8dc8-9de72ef96ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:34.431625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d4805-7963-4a3b-8dc8-9de72ef96ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:34.504767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-733d4805-7963-4a3b-8dc8-9de72ef96ba9 None None] MFA Rules not processed for user `14027a0e13fd4272bd7803e3131b6348`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:00:34.505927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-733d4805-7963-4a3b-8dc8-9de72ef96ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:34.579104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 676/1336] 10.222.0.94 () {58 vars in 1062 bytes} [Mon Jul 18 14:00:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:00:34.588166 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ab3b6-8cb7-4642-802c-6506847e03d9 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:00:34.589687 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3e7ab3b6-8cb7-4642-802c-6506847e03d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:34.629713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:34.694015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ab3b6-8cb7-4642-802c-6506847e03d9 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:00:34.770365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4770a40-8141-43f2-a709-0f229f1d85c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:00:34.770937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4770a40-8141-43f2-a709-0f229f1d85c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:34.771232 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4770a40-8141-43f2-a709-0f229f1d85c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:34.771537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4770a40-8141-43f2-a709-0f229f1d85c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:34.773296 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b4770a40-8141-43f2-a709-0f229f1d85c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:34.857926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:34.883766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4770a40-8141-43f2-a709-0f229f1d85c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:00:34.885243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4770a40-8141-43f2-a709-0f229f1d85c8 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:00:34.903656 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 661/1337] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:00:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:00:37.800268 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635bcb20-a5ea-4ac9-a7e6-724a2d7f5ba8 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:37.800880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635bcb20-a5ea-4ac9-a7e6-724a2d7f5ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:37.800880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635bcb20-a5ea-4ac9-a7e6-724a2d7f5ba8 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:37.802230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 677/1338] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:00:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:00:37.807514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2af71f-a40f-4866-8718-f26738b18cd2 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:37.807989 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2af71f-a40f-4866-8718-f26738b18cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:37.808102 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2af71f-a40f-4866-8718-f26738b18cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:37.871464 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-bd2af71f-a40f-4866-8718-f26738b18cd2 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:00:37.872695 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bd2af71f-a40f-4866-8718-f26738b18cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:37.968151 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 662/1339] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:00:37 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:00:38.005248 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635bcb20-a5ea-4ac9-a7e6-724a2d7f5ba8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:00:38.006769 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-635bcb20-a5ea-4ac9-a7e6-724a2d7f5ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:38.087370 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:38.117161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635bcb20-a5ea-4ac9-a7e6-724a2d7f5ba8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:00:38.189347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96cbe5-b492-498a-a1c2-53c0d641cca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:00:38.189989 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96cbe5-b492-498a-a1c2-53c0d641cca6 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:38.190276 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96cbe5-b492-498a-a1c2-53c0d641cca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:38.190556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96cbe5-b492-498a-a1c2-53c0d641cca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:38.192493 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-db96cbe5-b492-498a-a1c2-53c0d641cca6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:38.300153 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96cbe5-b492-498a-a1c2-53c0d641cca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:00:38.301586 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96cbe5-b492-498a-a1c2-53c0d641cca6 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:00:38.319476 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 678/1340] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:00:38 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:00:42.387798 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e57694-3157-4414-86f2-106a1f0639f0 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:42.388395 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e57694-3157-4414-86f2-106a1f0639f0 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:42.388525 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e57694-3157-4414-86f2-106a1f0639f0 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:42.389846 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 663/1341] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:00:42.395693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e9c12-7dc4-4700-aa77-bff10b6d4147 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:42.396124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e9c12-7dc4-4700-aa77-bff10b6d4147 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:42.396305 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e9c12-7dc4-4700-aa77-bff10b6d4147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:42.400440 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:42.464513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d43e9c12-7dc4-4700-aa77-bff10b6d4147 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:00:42.465715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d43e9c12-7dc4-4700-aa77-bff10b6d4147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:42.562988 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 679/1342] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:00:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:00:42.602076 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e57694-3157-4414-86f2-106a1f0639f0 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:00:42.603614 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-06e57694-3157-4414-86f2-106a1f0639f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:42.651677 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:42.707725 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e57694-3157-4414-86f2-106a1f0639f0 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:00:42.784290 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45ab29-df9b-4def-be03-5286205ca8cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:00:42.784948 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45ab29-df9b-4def-be03-5286205ca8cc service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:42.785239 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45ab29-df9b-4def-be03-5286205ca8cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:42.785525 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45ab29-df9b-4def-be03-5286205ca8cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:42.788141 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0a45ab29-df9b-4def-be03-5286205ca8cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:42.881377 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:42.898210 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45ab29-df9b-4def-be03-5286205ca8cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:00:42.899661 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45ab29-df9b-4def-be03-5286205ca8cc service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:00:42.917667 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 664/1343] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:00:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:00:45.530553 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46ea9a-23e1-48af-95e6-4e97a158ca50 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:45.531122 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46ea9a-23e1-48af-95e6-4e97a158ca50 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:45.531122 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46ea9a-23e1-48af-95e6-4e97a158ca50 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:45.532558 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 680/1344] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:00:45.538275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95925a91-82c9-462f-ae32-869841c402b7 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:45.538815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95925a91-82c9-462f-ae32-869841c402b7 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:45.538815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95925a91-82c9-462f-ae32-869841c402b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:45.607929 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-95925a91-82c9-462f-ae32-869841c402b7 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:00:45.609164 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-95925a91-82c9-462f-ae32-869841c402b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:45.708100 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 665/1345] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:00:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:00:45.745563 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46ea9a-23e1-48af-95e6-4e97a158ca50 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:00:45.747081 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2c46ea9a-23e1-48af-95e6-4e97a158ca50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:45.832328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:45.850080 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46ea9a-23e1-48af-95e6-4e97a158ca50 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:00:45.922317 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d33d0-5ef2-4fb2-841d-3cd433577240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:00:45.923564 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d33d0-5ef2-4fb2-841d-3cd433577240 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:45.923859 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d33d0-5ef2-4fb2-841d-3cd433577240 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:45.924166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d33d0-5ef2-4fb2-841d-3cd433577240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:45.925793 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ed3d33d0-5ef2-4fb2-841d-3cd433577240 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:46.036493 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d33d0-5ef2-4fb2-841d-3cd433577240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:00:46.038091 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d33d0-5ef2-4fb2-841d-3cd433577240 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:00:46.055857 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 681/1346] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:00:45 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:00:49.983844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc09d7d-963b-4582-ad71-7bde2375f242 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:49.984423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc09d7d-963b-4582-ad71-7bde2375f242 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:49.984423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc09d7d-963b-4582-ad71-7bde2375f242 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:49.986508 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 666/1347] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:00:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:00:49.992051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a2588-42bc-4264-b302-89a2c9dd1551 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:49.992491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a2588-42bc-4264-b302-89a2c9dd1551 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:49.992633 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a2588-42bc-4264-b302-89a2c9dd1551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:50.007571 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:50.059268 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9f6a2588-42bc-4264-b302-89a2c9dd1551 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:00:50.060607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9f6a2588-42bc-4264-b302-89a2c9dd1551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:50.159052 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 682/1348] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:00:50.198715 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc09d7d-963b-4582-ad71-7bde2375f242 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:00:50.200292 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-dfc09d7d-963b-4582-ad71-7bde2375f242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:50.286481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:50.336665 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc09d7d-963b-4582-ad71-7bde2375f242 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:00:50.417511 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a43c1-f3e8-4d5a-a58e-59e159414615 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:00:50.418274 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a43c1-f3e8-4d5a-a58e-59e159414615 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:50.418561 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a43c1-f3e8-4d5a-a58e-59e159414615 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:50.418840 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a43c1-f3e8-4d5a-a58e-59e159414615 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:50.420561 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b02a43c1-f3e8-4d5a-a58e-59e159414615 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:50.534918 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:50.538557 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a43c1-f3e8-4d5a-a58e-59e159414615 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:00:50.540587 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a43c1-f3e8-4d5a-a58e-59e159414615 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:00:50.559659 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 667/1349] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:00:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:00:55.461719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec0c9f-074e-4ea9-8dc5-a7eee409f28f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:55.462241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec0c9f-074e-4ea9-8dc5-a7eee409f28f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:55.462359 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec0c9f-074e-4ea9-8dc5-a7eee409f28f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:55.463661 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 683/1350] 10.222.0.94 () {54 vars in 921 bytes} [Mon Jul 18 14:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:00:55.469988 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe501fb-530b-47af-9bec-10f363f64d52 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:55.471603 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe501fb-530b-47af-9bec-10f363f64d52 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:55.472117 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe501fb-530b-47af-9bec-10f363f64d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:55.565245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-3fe501fb-530b-47af-9bec-10f363f64d52 None None] MFA Rules not processed for user `dd5b2ba9e37041c78a99f1700a4970c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:00:55.566641 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3fe501fb-530b-47af-9bec-10f363f64d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:00:55.679908 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 668/1351] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Jul 18 14:00:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:00:59.796763 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09c648-ab2c-4438-a0d0-94fe06a1447e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:59.797640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09c648-ab2c-4438-a0d0-94fe06a1447e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:59.797929 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09c648-ab2c-4438-a0d0-94fe06a1447e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:59.799318 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 684/1352] 10.222.0.94 () {54 vars in 921 bytes} [Mon Jul 18 14:00:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:00:59.804418 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40090616-e9ee-4c92-8bac-743876ea0939 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:00:59.804856 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40090616-e9ee-4c92-8bac-743876ea0939 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:00:59.805049 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40090616-e9ee-4c92-8bac-743876ea0939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:00:59.860335 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:00:59.885072 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-40090616-e9ee-4c92-8bac-743876ea0939 None None] MFA Rules not processed for user `dd5b2ba9e37041c78a99f1700a4970c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:00:59.886291 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-40090616-e9ee-4c92-8bac-743876ea0939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:00.006885 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 669/1353] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Jul 18 14:00:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:01:00.156909 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=59 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:01.274093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e21ef6-302a-461c-adb6-5d75fb6f1810 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:01.274715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e21ef6-302a-461c-adb6-5d75fb6f1810 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:01.274715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e21ef6-302a-461c-adb6-5d75fb6f1810 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:01.276048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 685/1354] 10.222.0.94 () {54 vars in 921 bytes} [Mon Jul 18 14:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:01:01.281024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f6e74-1d5a-4a05-8ab7-97b3adf3f2a5 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:01.281470 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f6e74-1d5a-4a05-8ab7-97b3adf3f2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:01.281570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f6e74-1d5a-4a05-8ab7-97b3adf3f2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:01.357192 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-272f6e74-1d5a-4a05-8ab7-97b3adf3f2a5 None None] MFA Rules not processed for user `dd5b2ba9e37041c78a99f1700a4970c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:01.358395 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-272f6e74-1d5a-4a05-8ab7-97b3adf3f2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:01.400797 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:01.466143 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 670/1355] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Jul 18 14:01:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:01:04.859808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a064525-2c37-4ccd-ac65-2c1b86f9af9a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:04.861011 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a064525-2c37-4ccd-ac65-2c1b86f9af9a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:04.861586 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a064525-2c37-4ccd-ac65-2c1b86f9af9a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:04.863509 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 686/1356] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:01:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:01:04.868952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8dc10f-16a7-4c04-9734-44113713de6a None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:04.869828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8dc10f-16a7-4c04-9734-44113713de6a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:04.870349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8dc10f-16a7-4c04-9734-44113713de6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:04.959330 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-3a8dc10f-16a7-4c04-9734-44113713de6a None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:04.959970 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3a8dc10f-16a7-4c04-9734-44113713de6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:05.069743 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 671/1357] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:01:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:01:05.118598 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a80de6-ac46-48e0-b86b-841c7c66cffa None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:05.119591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a80de6-ac46-48e0-b86b-841c7c66cffa None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:05.120151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a80de6-ac46-48e0-b86b-841c7c66cffa None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:05.122715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 687/1358] 10.222.0.94 () {54 vars in 962 bytes} [Mon Jul 18 14:01:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:01:05.131044 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f8a7d-2675-42b0-bd55-b27d80dae8bf None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:05.131717 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f8a7d-2675-42b0-bd55-b27d80dae8bf None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:05.132503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f8a7d-2675-42b0-bd55-b27d80dae8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:05.150640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:05.219627 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-709f8a7d-2675-42b0-bd55-b27d80dae8bf None None] MFA Rules not processed for user `ac717fcd9fc1499db12cf852d45572a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:05.220943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-709f8a7d-2675-42b0-bd55-b27d80dae8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:05.341159 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 672/1359] 10.222.0.94 () {58 vars in 1059 bytes} [Mon Jul 18 14:01:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:01:05.349204 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a80de6-ac46-48e0-b86b-841c7c66cffa None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:01:05.350722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-07a80de6-ac46-48e0-b86b-841c7c66cffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:05.497012 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a80de6-ac46-48e0-b86b-841c7c66cffa None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:01:05.576641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:05.620302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb31d96-8a5a-48ee-b067-9a08dfbf72e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:01:05.620864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb31d96-8a5a-48ee-b067-9a08dfbf72e5 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:05.621147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb31d96-8a5a-48ee-b067-9a08dfbf72e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:05.621426 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb31d96-8a5a-48ee-b067-9a08dfbf72e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:05.623055 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3cb31d96-8a5a-48ee-b067-9a08dfbf72e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:05.718734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb31d96-8a5a-48ee-b067-9a08dfbf72e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:01:05.720389 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb31d96-8a5a-48ee-b067-9a08dfbf72e5 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:01:05.740405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 688/1360] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:01:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:01:07.973494 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df7d99-d1b7-4d2f-98a6-0d951a3e6a76 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:07.973981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df7d99-d1b7-4d2f-98a6-0d951a3e6a76 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:07.974086 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df7d99-d1b7-4d2f-98a6-0d951a3e6a76 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:07.975607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 673/1361] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:01:07.980547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62793268-663c-43ea-a73e-e81bd08544e3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:07.980963 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62793268-663c-43ea-a73e-e81bd08544e3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:07.981172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62793268-663c-43ea-a73e-e81bd08544e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:08.051276 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-62793268-663c-43ea-a73e-e81bd08544e3 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:08.052571 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-62793268-663c-43ea-a73e-e81bd08544e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:08.092412 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:08.155012 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 689/1362] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:01:08.200177 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df7d99-d1b7-4d2f-98a6-0d951a3e6a76 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:01:08.201969 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-52df7d99-d1b7-4d2f-98a6-0d951a3e6a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:08.275300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:08.373123 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df7d99-d1b7-4d2f-98a6-0d951a3e6a76 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:01:08.499866 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb2cacf-55a2-4b1b-9ab9-af358962d9bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:01:08.500970 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2cacf-55a2-4b1b-9ab9-af358962d9bf service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:08.501259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2cacf-55a2-4b1b-9ab9-af358962d9bf service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:08.501259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2cacf-55a2-4b1b-9ab9-af358962d9bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:08.503126 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-aeb2cacf-55a2-4b1b-9ab9-af358962d9bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:08.534803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:08.617095 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb2cacf-55a2-4b1b-9ab9-af358962d9bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:01:08.619093 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb2cacf-55a2-4b1b-9ab9-af358962d9bf service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:01:08.638330 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 674/1363] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:01:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:01:10.970572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8bbbe-f22b-45eb-a31a-b53c56fe092b None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:10.971144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8bbbe-f22b-45eb-a31a-b53c56fe092b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:10.971144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8bbbe-f22b-45eb-a31a-b53c56fe092b None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:10.972547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 690/1364] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:01:10.978079 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83eaee-4872-430b-bd47-70a0181abc19 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:10.978899 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83eaee-4872-430b-bd47-70a0181abc19 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:10.979425 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83eaee-4872-430b-bd47-70a0181abc19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:11.056611 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-cc83eaee-4872-430b-bd47-70a0181abc19 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:11.057814 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-cc83eaee-4872-430b-bd47-70a0181abc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:11.179581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:11.183476 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 675/1365] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:01:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:01:11.230722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830cc7b-0a6c-44ae-8f52-cc9e8fd7fad1 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:11.231258 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830cc7b-0a6c-44ae-8f52-cc9e8fd7fad1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:11.231641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830cc7b-0a6c-44ae-8f52-cc9e8fd7fad1 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:11.233298 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 691/1366] 10.222.0.94 () {54 vars in 962 bytes} [Mon Jul 18 14:01:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:01:11.238826 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3cf65f-9199-44c9-90a0-0aa7a180a60e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:11.239218 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3cf65f-9199-44c9-90a0-0aa7a180a60e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:11.239618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3cf65f-9199-44c9-90a0-0aa7a180a60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:11.319223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-aa3cf65f-9199-44c9-90a0-0aa7a180a60e None None] MFA Rules not processed for user `ac717fcd9fc1499db12cf852d45572a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:11.320624 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-aa3cf65f-9199-44c9-90a0-0aa7a180a60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:11.443912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 676/1367] 10.222.0.94 () {58 vars in 1059 bytes} [Mon Jul 18 14:01:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:01:11.451777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830cc7b-0a6c-44ae-8f52-cc9e8fd7fad1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:01:11.453797 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1830cc7b-0a6c-44ae-8f52-cc9e8fd7fad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:11.616586 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830cc7b-0a6c-44ae-8f52-cc9e8fd7fad1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:01:11.638900 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:11.748072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa73329-7ded-41ed-a688-f106f9eeb5a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:01:11.749382 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa73329-7ded-41ed-a688-f106f9eeb5a4 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:11.749674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa73329-7ded-41ed-a688-f106f9eeb5a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:11.749961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa73329-7ded-41ed-a688-f106f9eeb5a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:11.751627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7fa73329-7ded-41ed-a688-f106f9eeb5a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:11.845131 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa73329-7ded-41ed-a688-f106f9eeb5a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:01:11.846770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa73329-7ded-41ed-a688-f106f9eeb5a4 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:01:11.881158 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 692/1368] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:01:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:01:12.986558 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9cdc63-61a6-4528-816b-4d5519524948 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:12.987102 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9cdc63-61a6-4528-816b-4d5519524948 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:12.987102 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9cdc63-61a6-4528-816b-4d5519524948 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:12.988693 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 677/1369] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:01:12.993826 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf3894-ad98-4fe6-a553-1affd06df641 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:12.994231 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf3894-ad98-4fe6-a553-1affd06df641 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:12.994397 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf3894-ad98-4fe6-a553-1affd06df641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:13.026549 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:13.066734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8daf3894-ad98-4fe6-a553-1affd06df641 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:13.068201 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8daf3894-ad98-4fe6-a553-1affd06df641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:13.192058 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 693/1370] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:01:13.541367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2982699-7734-43ec-a90e-df304f26d061 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:13.541664 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2982699-7734-43ec-a90e-df304f26d061 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:13.541950 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2982699-7734-43ec-a90e-df304f26d061 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:13.543554 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 678/1371] 10.222.0.94 () {54 vars in 964 bytes} [Mon Jul 18 14:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:01:13.549462 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb6ee4-812e-4bf8-a319-419020754ca8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:13.550125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb6ee4-812e-4bf8-a319-419020754ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:13.550125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb6ee4-812e-4bf8-a319-419020754ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:13.635146 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-87eb6ee4-812e-4bf8-a319-419020754ca8 None None] MFA Rules not processed for user `45ee15f7716541a1a61443069d5cf4f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:13.636400 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-87eb6ee4-812e-4bf8-a319-419020754ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:13.687602 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:13.719690 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 694/1372] 10.222.0.94 () {58 vars in 1061 bytes} [Mon Jul 18 14:01:13 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:01:13.726970 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2982699-7734-43ec-a90e-df304f26d061 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:01:13.728507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b2982699-7734-43ec-a90e-df304f26d061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:13.782726 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:13.844721 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2982699-7734-43ec-a90e-df304f26d061 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:01:13.919247 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2bffa-9dc5-4bef-9d6d-288f66fe8f44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:01:13.919907 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2bffa-9dc5-4bef-9d6d-288f66fe8f44 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:13.920257 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2bffa-9dc5-4bef-9d6d-288f66fe8f44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:13.920544 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2bffa-9dc5-4bef-9d6d-288f66fe8f44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:13.922188 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5aa2bffa-9dc5-4bef-9d6d-288f66fe8f44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:13.972134 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bc4a4-f36d-4bca-8b9a-2cfe285362ad None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:13.972475 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bc4a4-f36d-4bca-8b9a-2cfe285362ad None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:13.972789 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bc4a4-f36d-4bca-8b9a-2cfe285362ad None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:13.974342 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 695/1373] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:01:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:01:13.979574 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c39cd-daab-406d-934c-f18bee916d9a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:13.979878 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c39cd-daab-406d-934c-f18bee916d9a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:13.980195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c39cd-daab-406d-934c-f18bee916d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:14.034141 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:14.051300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2bffa-9dc5-4bef-9d6d-288f66fe8f44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:01:14.051488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2bffa-9dc5-4bef-9d6d-288f66fe8f44 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:01:14.056305 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c69c39cd-daab-406d-934c-f18bee916d9a None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:14.058082 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c69c39cd-daab-406d-934c-f18bee916d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:14.071537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 679/1374] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:01:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:01:14.168724 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 696/1375] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:01:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:01:14.215625 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2bffa-9dc5-4bef-9d6d-288f66fe8f44 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:01:14.218865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5aa2bffa-9dc5-4bef-9d6d-288f66fe8f44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:14.323598 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2bffa-9dc5-4bef-9d6d-288f66fe8f44 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:01:14.429541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:14.445804 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c6e40d-1684-4d1c-b376-1f2afb3aeb86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:01:14.446921 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6e40d-1684-4d1c-b376-1f2afb3aeb86 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:14.447435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6e40d-1684-4d1c-b376-1f2afb3aeb86 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:14.447930 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6e40d-1684-4d1c-b376-1f2afb3aeb86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:14.449825 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a6c6e40d-1684-4d1c-b376-1f2afb3aeb86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:14.543522 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c6e40d-1684-4d1c-b376-1f2afb3aeb86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:01:14.545936 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c6e40d-1684-4d1c-b376-1f2afb3aeb86 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:01:14.564841 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 680/1376] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:01:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:01:16.615815 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46882d89-48cf-4fc1-9e20-121515572fc0 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:16.616410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46882d89-48cf-4fc1-9e20-121515572fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:16.616410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46882d89-48cf-4fc1-9e20-121515572fc0 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:16.618081 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 697/1377] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:01:16.624074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d2452-c0d6-437f-9008-dfd9cb01a016 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:16.624520 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d2452-c0d6-437f-9008-dfd9cb01a016 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:16.624635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d2452-c0d6-437f-9008-dfd9cb01a016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:16.690503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-487d2452-c0d6-437f-9008-dfd9cb01a016 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:16.691881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-487d2452-c0d6-437f-9008-dfd9cb01a016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:16.750802 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:16.788488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 681/1378] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:01:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:01:16.834938 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46882d89-48cf-4fc1-9e20-121515572fc0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:01:16.836684 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-46882d89-48cf-4fc1-9e20-121515572fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:16.891290 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:16.995747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46882d89-48cf-4fc1-9e20-121515572fc0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:01:17.127388 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163634a0-4106-4f1b-9a5a-58d7e77eb7fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:01:17.128167 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163634a0-4106-4f1b-9a5a-58d7e77eb7fa service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:17.128451 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163634a0-4106-4f1b-9a5a-58d7e77eb7fa service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:17.128723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163634a0-4106-4f1b-9a5a-58d7e77eb7fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:17.130348 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-163634a0-4106-4f1b-9a5a-58d7e77eb7fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:17.141960 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:17.222638 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163634a0-4106-4f1b-9a5a-58d7e77eb7fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:01:17.224287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163634a0-4106-4f1b-9a5a-58d7e77eb7fa service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:01:17.242544 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 698/1379] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:01:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:01:19.308127 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77db60-3221-4d21-9011-989eca57fe9b None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:19.308601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77db60-3221-4d21-9011-989eca57fe9b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:19.308725 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77db60-3221-4d21-9011-989eca57fe9b None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:19.310220 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 682/1380] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:01:19.315490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b3d09-3079-4c6b-8f58-425605a3a4db None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:19.315906 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b3d09-3079-4c6b-8f58-425605a3a4db None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:19.316092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b3d09-3079-4c6b-8f58-425605a3a4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:19.383307 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-012b3d09-3079-4c6b-8f58-425605a3a4db None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:19.384537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-012b3d09-3079-4c6b-8f58-425605a3a4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:19.477571 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:19.496065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 699/1381] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:01:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:01:19.536895 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77db60-3221-4d21-9011-989eca57fe9b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:01:19.538404 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fe77db60-3221-4d21-9011-989eca57fe9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:19.711857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77db60-3221-4d21-9011-989eca57fe9b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:01:19.765660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:19.843076 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be211d7e-06a4-49d3-a9c8-85a36f4dd077 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:01:19.843746 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be211d7e-06a4-49d3-a9c8-85a36f4dd077 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:19.844065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be211d7e-06a4-49d3-a9c8-85a36f4dd077 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:19.844350 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be211d7e-06a4-49d3-a9c8-85a36f4dd077 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:19.845969 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-be211d7e-06a4-49d3-a9c8-85a36f4dd077 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:19.956600 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be211d7e-06a4-49d3-a9c8-85a36f4dd077 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:01:19.958018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be211d7e-06a4-49d3-a9c8-85a36f4dd077 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:01:19.977671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 683/1382] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:01:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:01:22.175237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bc38b-a8fa-4378-8eeb-aec8ab5c798f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:22.175804 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bc38b-a8fa-4378-8eeb-aec8ab5c798f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:22.176037 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bc38b-a8fa-4378-8eeb-aec8ab5c798f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:22.177903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 700/1383] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:01:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:01:22.185002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786fcdd-6d72-4472-9f75-0bd5d05d6dde None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:22.186041 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786fcdd-6d72-4472-9f75-0bd5d05d6dde None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:22.186182 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786fcdd-6d72-4472-9f75-0bd5d05d6dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:22.268700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-a786fcdd-6d72-4472-9f75-0bd5d05d6dde None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:22.270417 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a786fcdd-6d72-4472-9f75-0bd5d05d6dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:22.273259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:22.379297 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 684/1384] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:01:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:01:22.424311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312bc38b-a8fa-4378-8eeb-aec8ab5c798f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:01:22.425854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-312bc38b-a8fa-4378-8eeb-aec8ab5c798f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:22.579454 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312bc38b-a8fa-4378-8eeb-aec8ab5c798f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:01:22.650421 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:22.702197 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de5ee1-aa8d-4267-8d9b-5b9b1713b797 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:01:22.702844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de5ee1-aa8d-4267-8d9b-5b9b1713b797 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:22.703139 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de5ee1-aa8d-4267-8d9b-5b9b1713b797 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:22.703410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de5ee1-aa8d-4267-8d9b-5b9b1713b797 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:22.705105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-46de5ee1-aa8d-4267-8d9b-5b9b1713b797 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:22.797935 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de5ee1-aa8d-4267-8d9b-5b9b1713b797 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:01:22.799531 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de5ee1-aa8d-4267-8d9b-5b9b1713b797 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:01:22.818917 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 701/1385] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:01:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:01:24.904619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be26980-d854-4436-bef7-7ef9f608b039 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:24.905171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be26980-d854-4436-bef7-7ef9f608b039 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:24.905171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be26980-d854-4436-bef7-7ef9f608b039 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:24.906637 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 685/1386] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:01:24.912410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb41c6-2903-4398-97a1-c1eac01d8437 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:24.912794 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb41c6-2903-4398-97a1-c1eac01d8437 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:24.913158 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb41c6-2903-4398-97a1-c1eac01d8437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:24.992027 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4feb41c6-2903-4398-97a1-c1eac01d8437 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:24.993318 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4feb41c6-2903-4398-97a1-c1eac01d8437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:25.020138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:25.091774 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 702/1387] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:01:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:01:25.140166 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be26980-d854-4436-bef7-7ef9f608b039 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:01:25.141750 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0be26980-d854-4436-bef7-7ef9f608b039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:25.273054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:25.289889 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be26980-d854-4436-bef7-7ef9f608b039 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:01:25.402852 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9294aad-7aa6-4644-9ce9-e8063af20be7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:01:25.404052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9294aad-7aa6-4644-9ce9-e8063af20be7 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:25.404356 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9294aad-7aa6-4644-9ce9-e8063af20be7 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:25.404664 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9294aad-7aa6-4644-9ce9-e8063af20be7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:25.406363 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b9294aad-7aa6-4644-9ce9-e8063af20be7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:25.499146 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9294aad-7aa6-4644-9ce9-e8063af20be7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:01:25.500819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9294aad-7aa6-4644-9ce9-e8063af20be7 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:01:25.508060 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:25.523050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 686/1388] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:01:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:01:27.878097 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5251d-58b3-461f-b580-d7de08c1b58e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:27.878640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5251d-58b3-461f-b580-d7de08c1b58e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:27.878640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5251d-58b3-461f-b580-d7de08c1b58e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:27.880183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 703/1389] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:01:27.885371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702a047-ac50-4dd1-aa41-5ae874333232 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:27.885804 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702a047-ac50-4dd1-aa41-5ae874333232 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:27.885973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702a047-ac50-4dd1-aa41-5ae874333232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:27.954296 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-7702a047-ac50-4dd1-aa41-5ae874333232 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:27.955497 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7702a047-ac50-4dd1-aa41-5ae874333232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:28.052907 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 687/1390] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:01:28.097984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5251d-58b3-461f-b580-d7de08c1b58e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:01:28.099533 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d9c5251d-58b3-461f-b580-d7de08c1b58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:28.245992 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5251d-58b3-461f-b580-d7de08c1b58e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:01:28.256142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:28.363673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acffc92-d6c2-43d1-90d6-3774910e2724 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:01:28.364363 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acffc92-d6c2-43d1-90d6-3774910e2724 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:28.364655 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acffc92-d6c2-43d1-90d6-3774910e2724 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:28.364941 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acffc92-d6c2-43d1-90d6-3774910e2724 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:28.366578 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0acffc92-d6c2-43d1-90d6-3774910e2724 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:28.455898 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acffc92-d6c2-43d1-90d6-3774910e2724 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:01:28.457551 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acffc92-d6c2-43d1-90d6-3774910e2724 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:01:28.475541 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 704/1391] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:01:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:01:29.260816 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030c5bc-1572-463e-98b9-b7d38181a30a None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:29.261318 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030c5bc-1572-463e-98b9-b7d38181a30a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:29.261318 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030c5bc-1572-463e-98b9-b7d38181a30a None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:29.263312 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 688/1392] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:01:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:01:29.268675 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbea181-99cf-4f72-a3c2-ce1fefa8588b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:29.269065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbea181-99cf-4f72-a3c2-ce1fefa8588b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:29.269254 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbea181-99cf-4f72-a3c2-ce1fefa8588b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:29.283688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:29.336111 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9bbea181-99cf-4f72-a3c2-ce1fefa8588b None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:29.337384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9bbea181-99cf-4f72-a3c2-ce1fefa8588b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:29.430884 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 705/1393] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:01:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:01:29.486415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511db79-d735-4e33-b53c-66459ed5a22f None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:29.486704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511db79-d735-4e33-b53c-66459ed5a22f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:29.486982 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511db79-d735-4e33-b53c-66459ed5a22f None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:29.488372 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 689/1394] 10.222.0.94 () {54 vars in 961 bytes} [Mon Jul 18 14:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:01:29.493444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202a4fe-69ab-49e0-a96b-c3d702cd9a2c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:29.493739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202a4fe-69ab-49e0-a96b-c3d702cd9a2c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:29.494016 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202a4fe-69ab-49e0-a96b-c3d702cd9a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:29.568584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2202a4fe-69ab-49e0-a96b-c3d702cd9a2c None None] MFA Rules not processed for user `503a8228d4754c309fc3f365f4cfc61b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:29.569705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2202a4fe-69ab-49e0-a96b-c3d702cd9a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:29.609171 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:29.651925 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 706/1395] 10.222.0.94 () {58 vars in 1058 bytes} [Mon Jul 18 14:01:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:01:29.659385 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511db79-d735-4e33-b53c-66459ed5a22f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:01:29.660906 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d511db79-d735-4e33-b53c-66459ed5a22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:29.733970 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:29.764797 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511db79-d735-4e33-b53c-66459ed5a22f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:01:29.836633 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ecd30e-b7ca-411b-b9c3-3a437c1fa5f2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '503a8228d4754c309fc3f365f4cfc61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:01:29.837225 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecd30e-b7ca-411b-b9c3-3a437c1fa5f2 service glance] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:29.837514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecd30e-b7ca-411b-b9c3-3a437c1fa5f2 service glance] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:29.837794 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecd30e-b7ca-411b-b9c3-3a437c1fa5f2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:29.839414 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c0ecd30e-b7ca-411b-b9c3-3a437c1fa5f2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:29.948292 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ecd30e-b7ca-411b-b9c3-3a437c1fa5f2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:01:29.949925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ecd30e-b7ca-411b-b9c3-3a437c1fa5f2 service glance] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:01:29.965925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:29.969271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 690/1396] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:01:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:01:30.509928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80c492-61fd-4eb6-b0bb-7233193881c4 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:30.510431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80c492-61fd-4eb6-b0bb-7233193881c4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:30.510537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80c492-61fd-4eb6-b0bb-7233193881c4 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:30.511962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 707/1397] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:01:30.518236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63918f6c-f501-4407-b7ec-a56a69b3f022 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:30.518796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63918f6c-f501-4407-b7ec-a56a69b3f022 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:30.518796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63918f6c-f501-4407-b7ec-a56a69b3f022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:30.593732 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-63918f6c-f501-4407-b7ec-a56a69b3f022 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:30.594982 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-63918f6c-f501-4407-b7ec-a56a69b3f022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:30.672925 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1051762-e027-4d1f-953c-837f1600f958 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:30.673159 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1051762-e027-4d1f-953c-837f1600f958 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:30.673898 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1051762-e027-4d1f-953c-837f1600f958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:30.698815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 691/1398] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:01:30.741664 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d1051762-e027-4d1f-953c-837f1600f958 None None] MFA Rules not processed for user `94d824bffc2c4af299c537056bc21734`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:30.743091 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d1051762-e027-4d1f-953c-837f1600f958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:30.743326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63918f6c-f501-4407-b7ec-a56a69b3f022 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:01:30.744555 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-63918f6c-f501-4407-b7ec-a56a69b3f022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:30.819380 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:30.831692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 708/1399] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Jul 18 14:01:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:01:30.859329 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63918f6c-f501-4407-b7ec-a56a69b3f022 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:01:30.874951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83122afc-6caa-4019-bd85-7a23dc838a25 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:30.875331 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83122afc-6caa-4019-bd85-7a23dc838a25 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:30.875617 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83122afc-6caa-4019-bd85-7a23dc838a25 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:30.877038 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 709/1400] 10.222.0.94 () {54 vars in 943 bytes} [Mon Jul 18 14:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:01:30.883272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a0a18-2521-4e14-83d0-d6bb7a5f29ce None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:30.883560 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a0a18-2521-4e14-83d0-d6bb7a5f29ce None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:30.883838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a0a18-2521-4e14-83d0-d6bb7a5f29ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:30.947764 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:30.961848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b09a0a18-2521-4e14-83d0-d6bb7a5f29ce None None] MFA Rules not processed for user `cbf513ee25374e49b3d8ce28843e393a`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:30.963068 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b09a0a18-2521-4e14-83d0-d6bb7a5f29ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:30.988558 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fb98d-3274-4d27-9356-7187bafe0aa3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:01:30.989217 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fb98d-3274-4d27-9356-7187bafe0aa3 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:30.989507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fb98d-3274-4d27-9356-7187bafe0aa3 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:30.989790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fb98d-3274-4d27-9356-7187bafe0aa3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:30.991436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c56fb98d-3274-4d27-9356-7187bafe0aa3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:31.065268 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:31.072904 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 710/1401] 10.222.0.94 () {58 vars in 1040 bytes} [Mon Jul 18 14:01:30 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:01:31.082780 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a0a18-2521-4e14-83d0-d6bb7a5f29ce None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:01:31.082983 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56fb98d-3274-4d27-9356-7187bafe0aa3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:01:31.084647 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56fb98d-3274-4d27-9356-7187bafe0aa3 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:01:31.092774 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a0a18-2521-4e14-83d0-d6bb7a5f29ce None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:01:31.104300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 692/1402] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:01:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:01:31.207445 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:31.213768 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d1da7-85ae-4a30-abfe-30390a7834db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbf513ee25374e49b3d8ce28843e393a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:01:31.214442 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d1da7-85ae-4a30-abfe-30390a7834db service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:31.214731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d1da7-85ae-4a30-abfe-30390a7834db service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:31.215019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d1da7-85ae-4a30-abfe-30390a7834db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:31.217394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-943d1da7-85ae-4a30-abfe-30390a7834db service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:31.308908 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d1da7-85ae-4a30-abfe-30390a7834db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:01:31.311107 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d1da7-85ae-4a30-abfe-30390a7834db service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:01:31.319925 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 711/1403] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Jul 18 14:01:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 14:01:33.162281 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7c37b-5101-409c-b715-28dc8c2284b6 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:33.162842 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7c37b-5101-409c-b715-28dc8c2284b6 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:33.163256 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7c37b-5101-409c-b715-28dc8c2284b6 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:33.168922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 693/1404] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:01:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:01:33.177225 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1ce03-b5d8-487e-93a1-9cf5caebee02 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:33.177225 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1ce03-b5d8-487e-93a1-9cf5caebee02 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:33.177771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1ce03-b5d8-487e-93a1-9cf5caebee02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:33.264084 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f2d1ce03-b5d8-487e-93a1-9cf5caebee02 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:33.265238 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f2d1ce03-b5d8-487e-93a1-9cf5caebee02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:33.391283 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:33.412146 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 712/1405] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:01:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:01:33.459055 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7c37b-5101-409c-b715-28dc8c2284b6 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:01:33.460848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-85d7c37b-5101-409c-b715-28dc8c2284b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:33.648860 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7c37b-5101-409c-b715-28dc8c2284b6 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:01:33.752802 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:33.824340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c44ed2-867f-4d6e-8ee4-0e625f23237d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:01:33.825543 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c44ed2-867f-4d6e-8ee4-0e625f23237d service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:33.825894 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c44ed2-867f-4d6e-8ee4-0e625f23237d service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:33.826245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c44ed2-867f-4d6e-8ee4-0e625f23237d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:33.828073 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-56c44ed2-867f-4d6e-8ee4-0e625f23237d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:33.931565 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c44ed2-867f-4d6e-8ee4-0e625f23237d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:01:33.933135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c44ed2-867f-4d6e-8ee4-0e625f23237d service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:01:33.954083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 694/1406] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:01:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:01:36.014085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188da6fe-7e94-41b2-8486-01cb9e33387d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:36.014652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188da6fe-7e94-41b2-8486-01cb9e33387d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:36.014652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188da6fe-7e94-41b2-8486-01cb9e33387d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:36.016017 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 713/1407] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:01:36.021290 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174740ce-0c58-4cd8-b5af-c02a08edfe99 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:36.021769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174740ce-0c58-4cd8-b5af-c02a08edfe99 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:36.021864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174740ce-0c58-4cd8-b5af-c02a08edfe99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:36.092798 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-174740ce-0c58-4cd8-b5af-c02a08edfe99 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:36.096281 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-174740ce-0c58-4cd8-b5af-c02a08edfe99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:36.105175 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:36.216053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 695/1408] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:01:36.251684 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188da6fe-7e94-41b2-8486-01cb9e33387d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:01:36.253306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-188da6fe-7e94-41b2-8486-01cb9e33387d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:36.415874 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188da6fe-7e94-41b2-8486-01cb9e33387d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:01:36.486206 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:36.540016 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b074c-7522-41ab-8d8b-687a4f36341a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:01:36.540739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b074c-7522-41ab-8d8b-687a4f36341a service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:36.541025 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b074c-7522-41ab-8d8b-687a4f36341a service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:36.541301 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b074c-7522-41ab-8d8b-687a4f36341a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:36.543037 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-425b074c-7522-41ab-8d8b-687a4f36341a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:36.645255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b074c-7522-41ab-8d8b-687a4f36341a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:01:36.646960 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b074c-7522-41ab-8d8b-687a4f36341a service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:01:36.668470 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 714/1409] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:01:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:01:46.203789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=39 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:46.664657 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=67 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:53.436195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e09df-84a7-448d-b5e7-283b5370ba7d None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:53.437032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e09df-84a7-448d-b5e7-283b5370ba7d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:53.437032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e09df-84a7-448d-b5e7-283b5370ba7d None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:53.438425 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 696/1410] 10.222.0.94 () {54 vars in 918 bytes} [Mon Jul 18 14:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:01:53.443439 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdea3c-c1c7-42c1-8d2d-41b5b4a82351 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:53.443877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdea3c-c1c7-42c1-8d2d-41b5b4a82351 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:53.444040 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdea3c-c1c7-42c1-8d2d-41b5b4a82351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:53.518802 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-77bdea3c-c1c7-42c1-8d2d-41b5b4a82351 None None] MFA Rules not processed for user `dd5b2ba9e37041c78a99f1700a4970c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:53.520050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-77bdea3c-c1c7-42c1-8d2d-41b5b4a82351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:53.626324 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 715/1411] 10.222.0.94 () {58 vars in 1015 bytes} [Mon Jul 18 14:01:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:01:53.640944 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c0631-1199-4a3c-b628-2a37a63e5889 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:53.641240 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c0631-1199-4a3c-b628-2a37a63e5889 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:53.641527 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c0631-1199-4a3c-b628-2a37a63e5889 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:53.643022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 697/1412] 10.222.0.94 () {54 vars in 918 bytes} [Mon Jul 18 14:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:01:53.647674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ad806-5ec9-42e0-a946-688f93cd955f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:53.647999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ad806-5ec9-42e0-a946-688f93cd955f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:53.648290 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ad806-5ec9-42e0-a946-688f93cd955f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:53.723397 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:53.726381 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-192ad806-5ec9-42e0-a946-688f93cd955f None None] MFA Rules not processed for user `ac717fcd9fc1499db12cf852d45572a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:53.728143 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-192ad806-5ec9-42e0-a946-688f93cd955f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:53.846895 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 716/1413] 10.222.0.94 () {58 vars in 1015 bytes} [Mon Jul 18 14:01:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:01:56.305771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2a9a0-d1ba-4220-90cb-3bd1384098cb None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:56.306278 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2a9a0-d1ba-4220-90cb-3bd1384098cb None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:56.306278 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2a9a0-d1ba-4220-90cb-3bd1384098cb None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:56.307710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 698/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:01:56.313219 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2524855-4790-489a-b097-f4b1ecbbe97c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:56.313653 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2524855-4790-489a-b097-f4b1ecbbe97c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:56.313798 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2524855-4790-489a-b097-f4b1ecbbe97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:56.378128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c2524855-4790-489a-b097-f4b1ecbbe97c None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:56.379359 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c2524855-4790-489a-b097-f4b1ecbbe97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:56.435809 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:56.478362 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 717/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:01:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:01:56.521626 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2a9a0-d1ba-4220-90cb-3bd1384098cb None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:01:56.523135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-36f2a9a0-d1ba-4220-90cb-3bd1384098cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:56.667042 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2a9a0-d1ba-4220-90cb-3bd1384098cb None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:01:56.738193 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:56.778014 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df4223-b4b7-41c2-b138-0dcc5b53d76a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:01:56.778656 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df4223-b4b7-41c2-b138-0dcc5b53d76a service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:56.778938 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df4223-b4b7-41c2-b138-0dcc5b53d76a service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:56.779211 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df4223-b4b7-41c2-b138-0dcc5b53d76a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:56.780873 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-36df4223-b4b7-41c2-b138-0dcc5b53d76a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:56.870275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df4223-b4b7-41c2-b138-0dcc5b53d76a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:01:56.871888 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df4223-b4b7-41c2-b138-0dcc5b53d76a service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:01:56.889523 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 699/1416] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:01:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:01:57.787390 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283a8b5-a8fb-4ae7-b9c3-4dbc6fa3eb53 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:57.788013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283a8b5-a8fb-4ae7-b9c3-4dbc6fa3eb53 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:57.788013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283a8b5-a8fb-4ae7-b9c3-4dbc6fa3eb53 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:57.789375 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 718/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:01:57.794968 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f84228-c1eb-4341-93a1-b288df1af642 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:57.795390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f84228-c1eb-4341-93a1-b288df1af642 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:57.795538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f84228-c1eb-4341-93a1-b288df1af642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:57.868013 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-91f84228-c1eb-4341-93a1-b288df1af642 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:57.869255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-91f84228-c1eb-4341-93a1-b288df1af642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:57.895097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:57.965147 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 700/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:01:58.114985 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283a8b5-a8fb-4ae7-b9c3-4dbc6fa3eb53 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:01:58.116470 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2283a8b5-a8fb-4ae7-b9c3-4dbc6fa3eb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:58.258588 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283a8b5-a8fb-4ae7-b9c3-4dbc6fa3eb53 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:01:58.298080 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:58.369402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ece59-a8c9-4d21-ba34-5f8f0803627d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:01:58.370053 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ece59-a8c9-4d21-ba34-5f8f0803627d service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:58.370341 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ece59-a8c9-4d21-ba34-5f8f0803627d service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:58.370618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ece59-a8c9-4d21-ba34-5f8f0803627d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:58.372288 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-540ece59-a8c9-4d21-ba34-5f8f0803627d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:58.458825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ece59-a8c9-4d21-ba34-5f8f0803627d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:01:58.460474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ece59-a8c9-4d21-ba34-5f8f0803627d service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:01:58.479008 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 719/1419] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:01:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:01:59.376913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b984a-1b1a-4a44-83ad-104ae1222d7c None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:59.377447 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b984a-1b1a-4a44-83ad-104ae1222d7c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:59.377544 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b984a-1b1a-4a44-83ad-104ae1222d7c None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:59.378864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 701/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:01:59.384020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71211e-c4c0-4172-8855-0b0dc2cbef40 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:59.384477 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71211e-c4c0-4172-8855-0b0dc2cbef40 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:59.385234 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a71211e-c4c0-4172-8855-0b0dc2cbef40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:59.437493 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:59.460121 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6a71211e-c4c0-4172-8855-0b0dc2cbef40 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:01:59.461274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6a71211e-c4c0-4172-8855-0b0dc2cbef40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:59.557393 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 720/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:01:59.708206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b984a-1b1a-4a44-83ad-104ae1222d7c None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:01:59.709728 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d30b984a-1b1a-4a44-83ad-104ae1222d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:01:59.855142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b984a-1b1a-4a44-83ad-104ae1222d7c None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:01:59.867147 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:01:59.967957 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf942276-1812-4415-97a6-ff3f92211aba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:01:59.968618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf942276-1812-4415-97a6-ff3f92211aba service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:01:59.968904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf942276-1812-4415-97a6-ff3f92211aba service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:01:59.969183 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf942276-1812-4415-97a6-ff3f92211aba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:01:59.970807 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-cf942276-1812-4415-97a6-ff3f92211aba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:00.057575 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf942276-1812-4415-97a6-ff3f92211aba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:02:00.059204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf942276-1812-4415-97a6-ff3f92211aba service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:02:00.077598 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 702/1422] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:01:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:02:00.929549 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24e0a9-4a12-4b8a-867c-dba908cc3721 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:00.930095 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24e0a9-4a12-4b8a-867c-dba908cc3721 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:00.930095 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24e0a9-4a12-4b8a-867c-dba908cc3721 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:00.932262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 721/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:02:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:02:00.937505 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15bc93-007a-4f04-9aaf-f47411ff40d3 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:00.937915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15bc93-007a-4f04-9aaf-f47411ff40d3 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:00.938072 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15bc93-007a-4f04-9aaf-f47411ff40d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:00.957873 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:01.012638 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ad15bc93-007a-4f04-9aaf-f47411ff40d3 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:02:01.013829 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ad15bc93-007a-4f04-9aaf-f47411ff40d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:01.108548 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 703/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:02:01.258895 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24e0a9-4a12-4b8a-867c-dba908cc3721 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:02:01.260503 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ff24e0a9-4a12-4b8a-867c-dba908cc3721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:01.370035 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:01.404909 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24e0a9-4a12-4b8a-867c-dba908cc3721 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:02:01.515653 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370f4a8-3682-4082-b43f-fa93e4623f36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:02:01.516412 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370f4a8-3682-4082-b43f-fa93e4623f36 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:01.516693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370f4a8-3682-4082-b43f-fa93e4623f36 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:01.516969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370f4a8-3682-4082-b43f-fa93e4623f36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:01.518610 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6370f4a8-3682-4082-b43f-fa93e4623f36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:01.595631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:01.609884 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370f4a8-3682-4082-b43f-fa93e4623f36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:02:01.611576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370f4a8-3682-4082-b43f-fa93e4623f36 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:02:01.629433 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 722/1425] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:02:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:02:02.491115 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afab985-9e11-4a22-9196-755ad90fdeda None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:02.491657 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afab985-9e11-4a22-9196-755ad90fdeda None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:02.491777 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afab985-9e11-4a22-9196-755ad90fdeda None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:02.493089 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 704/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:02:02.498709 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d4fc2-fc37-41db-9e33-8063e7538e75 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:02.499105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d4fc2-fc37-41db-9e33-8063e7538e75 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:02.499281 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d4fc2-fc37-41db-9e33-8063e7538e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:02.573372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-851d4fc2-fc37-41db-9e33-8063e7538e75 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:02:02.574599 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-851d4fc2-fc37-41db-9e33-8063e7538e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:02.668886 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 723/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:02:02.819714 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afab985-9e11-4a22-9196-755ad90fdeda None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:02:02.821178 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4afab985-9e11-4a22-9196-755ad90fdeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:02.904600 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:02.969598 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afab985-9e11-4a22-9196-755ad90fdeda None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:02:03.079843 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc115ec-0b6a-4c24-9196-cd71a6b7ef85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:02:03.080467 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc115ec-0b6a-4c24-9196-cd71a6b7ef85 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:03.080760 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc115ec-0b6a-4c24-9196-cd71a6b7ef85 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:03.081041 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc115ec-0b6a-4c24-9196-cd71a6b7ef85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:03.082665 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-dcc115ec-0b6a-4c24-9196-cd71a6b7ef85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:03.128877 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:03.169408 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc115ec-0b6a-4c24-9196-cd71a6b7ef85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:02:03.171011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc115ec-0b6a-4c24-9196-cd71a6b7ef85 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:02:03.189314 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 705/1428] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:02:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:02:03.980585 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195fa13-f3ec-4c81-a670-59d63614bc62 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:03.981091 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195fa13-f3ec-4c81-a670-59d63614bc62 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:03.981222 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2195fa13-f3ec-4c81-a670-59d63614bc62 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:03.982575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 724/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:02:03.987485 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc74630-875b-4eed-b34a-614dc39473cd None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:03.987885 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc74630-875b-4eed-b34a-614dc39473cd None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:03.988096 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc74630-875b-4eed-b34a-614dc39473cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:04.061910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ddc74630-875b-4eed-b34a-614dc39473cd None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:02:04.063103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ddc74630-875b-4eed-b34a-614dc39473cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:04.155810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 706/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:02:04.303514 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195fa13-f3ec-4c81-a670-59d63614bc62 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:02:04.305030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2195fa13-f3ec-4c81-a670-59d63614bc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:04.354295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:04.453272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2195fa13-f3ec-4c81-a670-59d63614bc62 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:02:04.567669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b47e6e-79a0-4448-b3a3-1d90c148230b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:02:04.568284 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b47e6e-79a0-4448-b3a3-1d90c148230b service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:04.568576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b47e6e-79a0-4448-b3a3-1d90c148230b service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:04.568854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b47e6e-79a0-4448-b3a3-1d90c148230b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:04.570549 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-16b47e6e-79a0-4448-b3a3-1d90c148230b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:04.578164 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:04.659768 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b47e6e-79a0-4448-b3a3-1d90c148230b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:02:04.661552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b47e6e-79a0-4448-b3a3-1d90c148230b service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:02:04.679442 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 725/1431] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:02:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:02:05.483449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526342b-98de-43ab-9a02-f369c0e6e4d3 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:05.484019 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526342b-98de-43ab-9a02-f369c0e6e4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:05.484019 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526342b-98de-43ab-9a02-f369c0e6e4d3 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:05.485385 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 707/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:02:05.490370 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65be08-af76-4f10-a397-372025956ffc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:05.490807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65be08-af76-4f10-a397-372025956ffc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:05.490936 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65be08-af76-4f10-a397-372025956ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:05.565270 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ef65be08-af76-4f10-a397-372025956ffc None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:02:05.566474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ef65be08-af76-4f10-a397-372025956ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:05.633543 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:05.662793 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 726/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:02:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:02:05.811710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526342b-98de-43ab-9a02-f369c0e6e4d3 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:02:05.813187 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2526342b-98de-43ab-9a02-f369c0e6e4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:05.830841 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:05.961604 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526342b-98de-43ab-9a02-f369c0e6e4d3 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:02:06.045773 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:06.072122 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f111c-003b-4647-8d93-c9e21b55c5a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:02:06.072712 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f111c-003b-4647-8d93-c9e21b55c5a7 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:06.072995 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f111c-003b-4647-8d93-c9e21b55c5a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:06.073268 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f111c-003b-4647-8d93-c9e21b55c5a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:06.074900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b80f111c-003b-4647-8d93-c9e21b55c5a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:06.162187 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f111c-003b-4647-8d93-c9e21b55c5a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:02:06.163795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f111c-003b-4647-8d93-c9e21b55c5a7 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:02:06.181774 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 708/1434] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:02:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:02:06.988313 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26231d5d-e1a6-4101-9be4-482451d1c730 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:06.988800 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26231d5d-e1a6-4101-9be4-482451d1c730 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:06.988908 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26231d5d-e1a6-4101-9be4-482451d1c730 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:06.990289 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 727/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:02:06.995287 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3226bc-511c-4412-af51-d2167d9e2b91 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:06.995740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3226bc-511c-4412-af51-d2167d9e2b91 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:06.995857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3226bc-511c-4412-af51-d2167d9e2b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:07.068281 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-5e3226bc-511c-4412-af51-d2167d9e2b91 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:02:07.069473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5e3226bc-511c-4412-af51-d2167d9e2b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:07.113730 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:07.170088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 709/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:02:07.320829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26231d5d-e1a6-4101-9be4-482451d1c730 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:02:07.322868 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-26231d5d-e1a6-4101-9be4-482451d1c730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:07.483843 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26231d5d-e1a6-4101-9be4-482451d1c730 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:02:07.537150 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:07.596769 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998c3d1-ec5e-41f1-aa2a-513c10746d61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:02:07.597423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998c3d1-ec5e-41f1-aa2a-513c10746d61 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:07.597713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998c3d1-ec5e-41f1-aa2a-513c10746d61 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:07.597994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998c3d1-ec5e-41f1-aa2a-513c10746d61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:07.599649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a998c3d1-ec5e-41f1-aa2a-513c10746d61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:07.688908 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a998c3d1-ec5e-41f1-aa2a-513c10746d61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:02:07.690501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a998c3d1-ec5e-41f1-aa2a-513c10746d61 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:02:07.708869 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 728/1437] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:02:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:02:08.502560 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c880ae-91a4-40ea-ae1e-66c3018ad502 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:08.503059 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c880ae-91a4-40ea-ae1e-66c3018ad502 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:08.503167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c880ae-91a4-40ea-ae1e-66c3018ad502 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:08.504572 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 710/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:02:08.510576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d99555-7be3-45fe-87cd-4114c0cbf520 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:08.511004 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d99555-7be3-45fe-87cd-4114c0cbf520 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:08.511205 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d99555-7be3-45fe-87cd-4114c0cbf520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:08.586418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-30d99555-7be3-45fe-87cd-4114c0cbf520 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:02:08.587626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-30d99555-7be3-45fe-87cd-4114c0cbf520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:08.589638 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:08.683676 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 729/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:02:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:02:08.833309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c880ae-91a4-40ea-ae1e-66c3018ad502 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:02:08.834775 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-61c880ae-91a4-40ea-ae1e-66c3018ad502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:08.983620 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c880ae-91a4-40ea-ae1e-66c3018ad502 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:02:09.009758 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:09.094370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a25967-dfe3-449c-85e3-a42c1bd9653d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:02:09.094945 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a25967-dfe3-449c-85e3-a42c1bd9653d service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:09.095229 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a25967-dfe3-449c-85e3-a42c1bd9653d service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:09.095503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a25967-dfe3-449c-85e3-a42c1bd9653d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:09.097142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-38a25967-dfe3-449c-85e3-a42c1bd9653d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:09.186172 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a25967-dfe3-449c-85e3-a42c1bd9653d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:02:09.187788 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a25967-dfe3-449c-85e3-a42c1bd9653d service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:02:09.207351 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 711/1440] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:02:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:02:11.093946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4ce3f-d89b-4b2c-9828-37b15ab4bd50 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:11.094494 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4ce3f-d89b-4b2c-9828-37b15ab4bd50 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:11.094494 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4ce3f-d89b-4b2c-9828-37b15ab4bd50 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:11.095893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 730/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:02:11.101304 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad354592-be65-4cb0-aae1-9e4453d7d997 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:11.101723 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad354592-be65-4cb0-aae1-9e4453d7d997 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:11.101881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad354592-be65-4cb0-aae1-9e4453d7d997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:11.143349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:11.168245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ad354592-be65-4cb0-aae1-9e4453d7d997 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:02:11.169459 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ad354592-be65-4cb0-aae1-9e4453d7d997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:11.265360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 712/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:02:11.303380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4ce3f-d89b-4b2c-9828-37b15ab4bd50 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:02:11.304956 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-57d4ce3f-d89b-4b2c-9828-37b15ab4bd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:11.407588 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4ce3f-d89b-4b2c-9828-37b15ab4bd50 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:02:11.439252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:11.481438 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8819373-6cbe-4eb2-95f1-b7b716cfe073 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:02:11.482134 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8819373-6cbe-4eb2-95f1-b7b716cfe073 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:11.482415 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8819373-6cbe-4eb2-95f1-b7b716cfe073 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:11.482689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8819373-6cbe-4eb2-95f1-b7b716cfe073 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:11.484366 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c8819373-6cbe-4eb2-95f1-b7b716cfe073 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:11.590900 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8819373-6cbe-4eb2-95f1-b7b716cfe073 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:02:11.592397 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8819373-6cbe-4eb2-95f1-b7b716cfe073 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:02:11.609838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 731/1443] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:02:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:02:13.817361 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa893b-afb4-4e25-bde9-2ff466c4b09b None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:13.817868 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa893b-afb4-4e25-bde9-2ff466c4b09b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:13.817868 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa893b-afb4-4e25-bde9-2ff466c4b09b None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:13.819201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 713/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:02:13.824976 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d684c50-b470-4bc7-a61f-a30dfeea8a98 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:13.825375 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d684c50-b470-4bc7-a61f-a30dfeea8a98 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:13.825544 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d684c50-b470-4bc7-a61f-a30dfeea8a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:13.888133 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5d684c50-b470-4bc7-a61f-a30dfeea8a98 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:02:13.889329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5d684c50-b470-4bc7-a61f-a30dfeea8a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:13.903396 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:13.987618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 732/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:02:14.026271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa893b-afb4-4e25-bde9-2ff466c4b09b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:02:14.027818 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ccaa893b-afb4-4e25-bde9-2ff466c4b09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:14.128649 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa893b-afb4-4e25-bde9-2ff466c4b09b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:02:14.141028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:14.202780 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24793152-83d4-4b46-aa48-1d24a05c488b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:02:14.203433 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24793152-83d4-4b46-aa48-1d24a05c488b service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:14.203724 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24793152-83d4-4b46-aa48-1d24a05c488b service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:14.204035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24793152-83d4-4b46-aa48-1d24a05c488b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:14.205709 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-24793152-83d4-4b46-aa48-1d24a05c488b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:14.321088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24793152-83d4-4b46-aa48-1d24a05c488b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:02:14.322541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24793152-83d4-4b46-aa48-1d24a05c488b service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:02:14.340199 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 714/1446] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:02:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:02:16.624629 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a88c74-d64e-48e2-ba30-182e4e48b925 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:16.625165 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a88c74-d64e-48e2-ba30-182e4e48b925 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:16.625283 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a88c74-d64e-48e2-ba30-182e4e48b925 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:16.626597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 733/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:02:16.631979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c0211-3e59-416d-9ae1-c7c90bccdf38 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:16.632414 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c0211-3e59-416d-9ae1-c7c90bccdf38 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:16.632569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c0211-3e59-416d-9ae1-c7c90bccdf38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:16.679229 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:16.698653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f55c0211-3e59-416d-9ae1-c7c90bccdf38 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:02:16.699936 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f55c0211-3e59-416d-9ae1-c7c90bccdf38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:16.793736 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 715/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:02:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:02:16.833957 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a88c74-d64e-48e2-ba30-182e4e48b925 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:02:16.835525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c9a88c74-d64e-48e2-ba30-182e4e48b925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:16.938181 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a88c74-d64e-48e2-ba30-182e4e48b925 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:02:16.979285 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:17.013159 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16f113-a4fb-4c1d-8b43-bb37bdd15aa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:02:17.013806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16f113-a4fb-4c1d-8b43-bb37bdd15aa2 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:17.014094 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16f113-a4fb-4c1d-8b43-bb37bdd15aa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:17.014372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16f113-a4fb-4c1d-8b43-bb37bdd15aa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:17.016064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3d16f113-a4fb-4c1d-8b43-bb37bdd15aa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:17.125091 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16f113-a4fb-4c1d-8b43-bb37bdd15aa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:02:17.126565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16f113-a4fb-4c1d-8b43-bb37bdd15aa2 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:02:17.144262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 734/1449] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:02:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:02:19.262119 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7214dd-73b5-4d25-a729-27c45a5036ad None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:19.262614 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7214dd-73b5-4d25-a729-27c45a5036ad None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:19.262721 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7214dd-73b5-4d25-a729-27c45a5036ad None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:19.264056 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 716/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:02:19.269354 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3b666-7fa8-4ab4-b1e8-7dfcd510e4bf None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:19.269800 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3b666-7fa8-4ab4-b1e8-7dfcd510e4bf None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:19.269910 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3b666-7fa8-4ab4-b1e8-7dfcd510e4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:19.333908 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-67f3b666-7fa8-4ab4-b1e8-7dfcd510e4bf None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:02:19.335136 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-67f3b666-7fa8-4ab4-b1e8-7dfcd510e4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:19.355658 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:19.431023 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 735/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:02:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:02:19.469666 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7214dd-73b5-4d25-a729-27c45a5036ad None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:02:19.471185 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5b7214dd-73b5-4d25-a729-27c45a5036ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:19.572922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7214dd-73b5-4d25-a729-27c45a5036ad None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:02:19.592966 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:19.646531 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3491f-44eb-4ef2-a04c-a47e43bcb879 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:02:19.647185 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3491f-44eb-4ef2-a04c-a47e43bcb879 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:19.647474 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3491f-44eb-4ef2-a04c-a47e43bcb879 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:19.647759 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3491f-44eb-4ef2-a04c-a47e43bcb879 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:19.649448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a6b3491f-44eb-4ef2-a04c-a47e43bcb879 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:19.760428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3491f-44eb-4ef2-a04c-a47e43bcb879 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:02:19.761788 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3491f-44eb-4ef2-a04c-a47e43bcb879 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:02:19.779418 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 717/1452] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:02:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:02:21.845833 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb3e64-8d78-4e43-9e75-55559ac213bd None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:21.846396 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb3e64-8d78-4e43-9e75-55559ac213bd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:21.846396 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb3e64-8d78-4e43-9e75-55559ac213bd None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:21.847778 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 736/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:02:21.853710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e310a13-2257-495a-82ba-57f43c35aeac None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:21.854092 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e310a13-2257-495a-82ba-57f43c35aeac None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:21.854281 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e310a13-2257-495a-82ba-57f43c35aeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:21.916327 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:21.919213 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-9e310a13-2257-495a-82ba-57f43c35aeac None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:02:21.920996 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9e310a13-2257-495a-82ba-57f43c35aeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:22.027267 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 718/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:02:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:02:22.064940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb3e64-8d78-4e43-9e75-55559ac213bd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:02:22.066480 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5ceb3e64-8d78-4e43-9e75-55559ac213bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:22.169146 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb3e64-8d78-4e43-9e75-55559ac213bd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:02:22.218310 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:22.242919 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48337ef1-9f48-4494-ab4d-56ca33de3876 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:02:22.243571 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48337ef1-9f48-4494-ab4d-56ca33de3876 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:22.243859 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48337ef1-9f48-4494-ab4d-56ca33de3876 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:22.244168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48337ef1-9f48-4494-ab4d-56ca33de3876 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:22.245806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-48337ef1-9f48-4494-ab4d-56ca33de3876 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:22.356117 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48337ef1-9f48-4494-ab4d-56ca33de3876 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:02:22.357571 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48337ef1-9f48-4494-ab4d-56ca33de3876 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:02:22.375962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 737/1455] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:02:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:02:24.472596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c5714-1a87-4681-b157-f792ae64a54d None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:24.473154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c5714-1a87-4681-b157-f792ae64a54d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:24.473154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c5714-1a87-4681-b157-f792ae64a54d None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:24.474556 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 719/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:02:24.480247 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f58b4-4201-4aa4-a5e8-d7449bf0978c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:24.480674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f58b4-4201-4aa4-a5e8-d7449bf0978c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:24.480829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f58b4-4201-4aa4-a5e8-d7449bf0978c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:24.547395 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f69f58b4-4201-4aa4-a5e8-d7449bf0978c None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:02:24.548663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f69f58b4-4201-4aa4-a5e8-d7449bf0978c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:24.577833 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:24.645988 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 738/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:02:24.688505 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c5714-1a87-4681-b157-f792ae64a54d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:02:24.690096 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fc0c5714-1a87-4681-b157-f792ae64a54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:24.800286 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c5714-1a87-4681-b157-f792ae64a54d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:02:24.829095 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:24.875175 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f03c6-844a-4d56-a306-523d07ab4db4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:02:24.875826 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f03c6-844a-4d56-a306-523d07ab4db4 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:24.876162 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f03c6-844a-4d56-a306-523d07ab4db4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:24.876444 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f03c6-844a-4d56-a306-523d07ab4db4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:24.878066 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3c2f03c6-844a-4d56-a306-523d07ab4db4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:24.985552 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f03c6-844a-4d56-a306-523d07ab4db4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:02:24.987010 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f03c6-844a-4d56-a306-523d07ab4db4 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:02:25.005366 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 720/1458] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:02:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:02:27.073291 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2efad9b-91e9-4399-8d4a-35ed8d1ca658 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:27.073869 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2efad9b-91e9-4399-8d4a-35ed8d1ca658 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:27.073869 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2efad9b-91e9-4399-8d4a-35ed8d1ca658 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:27.075304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 739/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:02:27.080669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ada63-2762-453f-a11b-46a84b303e56 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:27.081065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ada63-2762-453f-a11b-46a84b303e56 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:27.081170 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ada63-2762-453f-a11b-46a84b303e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:27.146127 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ae5ada63-2762-453f-a11b-46a84b303e56 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:02:27.147343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ae5ada63-2762-453f-a11b-46a84b303e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:27.156216 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:27.246196 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 721/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:02:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:02:27.284514 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2efad9b-91e9-4399-8d4a-35ed8d1ca658 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:02:27.286048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a2efad9b-91e9-4399-8d4a-35ed8d1ca658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:27.387134 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2efad9b-91e9-4399-8d4a-35ed8d1ca658 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:02:27.442024 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:27.460717 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b5d26-51c2-41ff-b4ad-77ce5150bc44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:02:27.461926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b5d26-51c2-41ff-b4ad-77ce5150bc44 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:27.462217 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b5d26-51c2-41ff-b4ad-77ce5150bc44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:27.462499 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b5d26-51c2-41ff-b4ad-77ce5150bc44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:27.464168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-529b5d26-51c2-41ff-b4ad-77ce5150bc44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:27.570166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b5d26-51c2-41ff-b4ad-77ce5150bc44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:02:27.572272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b5d26-51c2-41ff-b4ad-77ce5150bc44 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:02:27.590562 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 740/1461] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:02:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:02:29.634255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3171e71-949e-46cc-915c-1a89952f02c0 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:29.634772 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3171e71-949e-46cc-915c-1a89952f02c0 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:29.634892 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3171e71-949e-46cc-915c-1a89952f02c0 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:29.636239 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 722/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:02:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:02:29.641698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64210980-77e1-46da-8a26-9670ad7f1757 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:29.642121 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64210980-77e1-46da-8a26-9670ad7f1757 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:29.642275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64210980-77e1-46da-8a26-9670ad7f1757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:29.708756 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-64210980-77e1-46da-8a26-9670ad7f1757 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:02:29.709984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-64210980-77e1-46da-8a26-9670ad7f1757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:29.748553 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:29.807528 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 741/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:02:29.845619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3171e71-949e-46cc-915c-1a89952f02c0 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:02:29.847144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c3171e71-949e-46cc-915c-1a89952f02c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:29.950409 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3171e71-949e-46cc-915c-1a89952f02c0 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:02:29.986947 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:30.023219 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd6889-acf5-4acc-b756-26edc2fe4380 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:02:30.023929 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd6889-acf5-4acc-b756-26edc2fe4380 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:30.024245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd6889-acf5-4acc-b756-26edc2fe4380 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:30.024522 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd6889-acf5-4acc-b756-26edc2fe4380 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:30.026142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-00dd6889-acf5-4acc-b756-26edc2fe4380 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:30.137928 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd6889-acf5-4acc-b756-26edc2fe4380 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:02:30.139368 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd6889-acf5-4acc-b756-26edc2fe4380 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:02:30.157741 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 723/1464] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:02:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:02:32.181734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53def117-dafb-4401-bb15-44bd162cea55 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:32.182264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53def117-dafb-4401-bb15-44bd162cea55 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:32.182354 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53def117-dafb-4401-bb15-44bd162cea55 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:32.183779 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 742/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:02:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:02:32.189315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a01c0-1148-4ebd-a041-473af97d0bd5 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:32.189739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a01c0-1148-4ebd-a041-473af97d0bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:32.189880 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a01c0-1148-4ebd-a041-473af97d0bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:32.254061 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-304a01c0-1148-4ebd-a041-473af97d0bd5 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:02:32.255195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-304a01c0-1148-4ebd-a041-473af97d0bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:32.273261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:32.351349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 724/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:02:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:02:32.388430 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53def117-dafb-4401-bb15-44bd162cea55 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:02:32.390019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-53def117-dafb-4401-bb15-44bd162cea55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:32.497031 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53def117-dafb-4401-bb15-44bd162cea55 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:02:32.563893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:32.571260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080229e-2ff5-46eb-b19c-8678ad782d52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:02:32.571910 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080229e-2ff5-46eb-b19c-8678ad782d52 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:32.572230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080229e-2ff5-46eb-b19c-8678ad782d52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:32.572508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080229e-2ff5-46eb-b19c-8678ad782d52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:32.574138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1080229e-2ff5-46eb-b19c-8678ad782d52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:32.681147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1080229e-2ff5-46eb-b19c-8678ad782d52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:02:32.682606 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1080229e-2ff5-46eb-b19c-8678ad782d52 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:02:32.700429 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 743/1467] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:02:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:02:34.726386 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa909abd-53e0-4999-90f4-17904d3d35aa None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:34.726948 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa909abd-53e0-4999-90f4-17904d3d35aa None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:34.726948 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa909abd-53e0-4999-90f4-17904d3d35aa None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:34.729014 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 725/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:02:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:02:34.734217 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a92aa-cd42-4277-be27-53c64043f9a3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:34.734638 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a92aa-cd42-4277-be27-53c64043f9a3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:34.734804 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a92aa-cd42-4277-be27-53c64043f9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:34.798672 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ff4a92aa-cd42-4277-be27-53c64043f9a3 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:02:34.799891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ff4a92aa-cd42-4277-be27-53c64043f9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:34.849197 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:34.896947 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 744/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:02:34.934034 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa909abd-53e0-4999-90f4-17904d3d35aa None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:02:34.935573 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-aa909abd-53e0-4999-90f4-17904d3d35aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:35.038016 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa909abd-53e0-4999-90f4-17904d3d35aa None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:02:35.084867 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:35.111710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043075ce-ffa3-4fcc-aef9-ae773b8002a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:02:35.112406 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043075ce-ffa3-4fcc-aef9-ae773b8002a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:35.112698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043075ce-ffa3-4fcc-aef9-ae773b8002a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:35.112981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043075ce-ffa3-4fcc-aef9-ae773b8002a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:35.114649 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-043075ce-ffa3-4fcc-aef9-ae773b8002a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:35.226103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043075ce-ffa3-4fcc-aef9-ae773b8002a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:02:35.227597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043075ce-ffa3-4fcc-aef9-ae773b8002a6 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:02:35.246611 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 726/1470] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:02:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:02:44.895126 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=22 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:45.244069 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=66 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:54.828384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3bf2c-0ba0-48b2-9974-173a9674c4c2 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:54.828908 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3bf2c-0ba0-48b2-9974-173a9674c4c2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:54.828908 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3bf2c-0ba0-48b2-9974-173a9674c4c2 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:54.830272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 745/1471] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:02:54.835438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37168b-5c62-49c2-a03a-c014747d2bf7 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:54.835868 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37168b-5c62-49c2-a03a-c014747d2bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:54.836099 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37168b-5c62-49c2-a03a-c014747d2bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:54.900536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-7a37168b-5c62-49c2-a03a-c014747d2bf7 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:02:54.901760 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7a37168b-5c62-49c2-a03a-c014747d2bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:54.997656 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 727/1472] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:02:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:02:55.045862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1ed83-00e8-4b8b-8965-b0c12cf016fc None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:55.046160 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1ed83-00e8-4b8b-8965-b0c12cf016fc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:55.046448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1ed83-00e8-4b8b-8965-b0c12cf016fc None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:55.047821 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 746/1473] 10.222.0.94 () {54 vars in 961 bytes} [Mon Jul 18 14:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:02:55.052672 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76495d-544b-4560-b314-dbbbd58e3606 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:55.052968 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76495d-544b-4560-b314-dbbbd58e3606 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:55.053250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76495d-544b-4560-b314-dbbbd58e3606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:55.131563 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-dc76495d-544b-4560-b314-dbbbd58e3606 None None] MFA Rules not processed for user `503a8228d4754c309fc3f365f4cfc61b`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:02:55.132820 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-dc76495d-544b-4560-b314-dbbbd58e3606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:55.156474 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:55.212284 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 728/1474] 10.222.0.94 () {58 vars in 1058 bytes} [Mon Jul 18 14:02:55 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:02:55.219782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d1ed83-00e8-4b8b-8965-b0c12cf016fc None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:02:55.221340 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-90d1ed83-00e8-4b8b-8965-b0c12cf016fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:55.323869 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d1ed83-00e8-4b8b-8965-b0c12cf016fc None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:02:55.396572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a98bff1-99e8-4d32-a926-70e2e64f1975 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '503a8228d4754c309fc3f365f4cfc61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:02:55.397148 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98bff1-99e8-4d32-a926-70e2e64f1975 service glance] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:55.397437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98bff1-99e8-4d32-a926-70e2e64f1975 service glance] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:55.397720 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98bff1-99e8-4d32-a926-70e2e64f1975 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:55.399339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4a98bff1-99e8-4d32-a926-70e2e64f1975 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:55.455114 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:55.509435 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a98bff1-99e8-4d32-a926-70e2e64f1975 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:02:55.510899 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a98bff1-99e8-4d32-a926-70e2e64f1975 service glance] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:02:55.531324 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 747/1475] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:02:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:02:57.659179 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206287d5-5cae-4c2f-be74-58164e387728 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:57.659720 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206287d5-5cae-4c2f-be74-58164e387728 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:57.659720 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206287d5-5cae-4c2f-be74-58164e387728 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:57.661202 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 729/1476] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:02:57.667615 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa96012-fea0-4335-bf79-8bf900c066a3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:57.668077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa96012-fea0-4335-bf79-8bf900c066a3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:57.668219 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa96012-fea0-4335-bf79-8bf900c066a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:57.732322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2fa96012-fea0-4335-bf79-8bf900c066a3 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:02:57.733526 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2fa96012-fea0-4335-bf79-8bf900c066a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:57.828567 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 748/1477] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:02:57.851378 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206287d5-5cae-4c2f-be74-58164e387728 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:02:57.853023 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-206287d5-5cae-4c2f-be74-58164e387728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:02:57.977187 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206287d5-5cae-4c2f-be74-58164e387728 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:02:58.026355 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:02:58.070230 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba322f88-790c-4d01-a334-c5a751466b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:02:58.070894 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba322f88-790c-4d01-a334-c5a751466b88 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:02:58.071182 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba322f88-790c-4d01-a334-c5a751466b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:02:58.071461 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba322f88-790c-4d01-a334-c5a751466b88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:02:58.072711 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba322f88-790c-4d01-a334-c5a751466b88 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:02:58.074479 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: WARNING py.warnings [None req-ba322f88-790c-4d01-a334-c5a751466b88 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:02:58.074479 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: warnings.warn(msg) Jul 18 14:02:58.074479 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: Jul 18 14:02:58.075643 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba322f88-790c-4d01-a334-c5a751466b88 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:02:58.094691 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 730/1478] 10.222.0.94 () {56 vars in 1108 bytes} [Mon Jul 18 14:02:57 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:03:00.091170 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a2617-0844-4eeb-b97f-2de016e6f9c0 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:03:00.091739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a2617-0844-4eeb-b97f-2de016e6f9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:03:00.091739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a2617-0844-4eeb-b97f-2de016e6f9c0 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:03:00.093120 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 749/1479] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:03:00.098352 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba25514-4e92-436c-a783-b94587529726 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:03:00.098798 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba25514-4e92-436c-a783-b94587529726 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:03:00.098931 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba25514-4e92-436c-a783-b94587529726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:03:00.163693 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-1ba25514-4e92-436c-a783-b94587529726 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:03:00.164936 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1ba25514-4e92-436c-a783-b94587529726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:03:00.262309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 731/1480] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:03:00.305076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a2617-0844-4eeb-b97f-2de016e6f9c0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:03:00.306954 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a71a2617-0844-4eeb-b97f-2de016e6f9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:03:00.317261 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:03:00.577007 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a2617-0844-4eeb-b97f-2de016e6f9c0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:03:00.656556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:03:00.689688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5798acfe-4fe6-4454-ab2f-f81a0b0747d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:03:00.690340 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798acfe-4fe6-4454-ab2f-f81a0b0747d2 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:03:00.690629 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798acfe-4fe6-4454-ab2f-f81a0b0747d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:03:00.690912 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798acfe-4fe6-4454-ab2f-f81a0b0747d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:03:00.692566 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5798acfe-4fe6-4454-ab2f-f81a0b0747d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:03:00.779015 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5798acfe-4fe6-4454-ab2f-f81a0b0747d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:03:00.780671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5798acfe-4fe6-4454-ab2f-f81a0b0747d2 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:03:00.798340 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 750/1481] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:03:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:03:02.991356 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a1818-7b5d-4a92-8746-e9c5a12b5d9a None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:03:02.991855 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a1818-7b5d-4a92-8746-e9c5a12b5d9a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:03:02.991978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a1818-7b5d-4a92-8746-e9c5a12b5d9a None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:03:02.993303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 732/1482] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:03:02.999073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86fa2a-8622-4dcd-8fd7-24d6e98f07ab None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:03:02.999471 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86fa2a-8622-4dcd-8fd7-24d6e98f07ab None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:03:02.999646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86fa2a-8622-4dcd-8fd7-24d6e98f07ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:03:03.063498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3f86fa2a-8622-4dcd-8fd7-24d6e98f07ab None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:03:03.064747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3f86fa2a-8622-4dcd-8fd7-24d6e98f07ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:03:03.110353 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:03:03.160196 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 751/1483] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:03:03.198325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a1818-7b5d-4a92-8746-e9c5a12b5d9a None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:03:03.199885 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b24a1818-7b5d-4a92-8746-e9c5a12b5d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:03:03.223218 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:03:03.303710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a1818-7b5d-4a92-8746-e9c5a12b5d9a None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:03:03.375974 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c965f-2c99-44ce-8dcf-14b00afb8e77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:03:03.376619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c965f-2c99-44ce-8dcf-14b00afb8e77 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:03:03.376910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c965f-2c99-44ce-8dcf-14b00afb8e77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:03:03.377193 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c965f-2c99-44ce-8dcf-14b00afb8e77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:03:03.378844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-270c965f-2c99-44ce-8dcf-14b00afb8e77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:03:03.451675 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:03:03.486669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c965f-2c99-44ce-8dcf-14b00afb8e77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:03:03.488143 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c965f-2c99-44ce-8dcf-14b00afb8e77 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:03:03.506401 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 733/1484] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:03:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:03:05.581569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645b295-9230-431d-8775-451dffb5c2b6 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:03:05.582018 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645b295-9230-431d-8775-451dffb5c2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:03:05.582142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645b295-9230-431d-8775-451dffb5c2b6 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:03:05.583541 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 752/1485] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:03:05.589364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ded4d-4aab-4c7b-a432-b52a64767784 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:03:05.589752 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ded4d-4aab-4c7b-a432-b52a64767784 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:03:05.589925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ded4d-4aab-4c7b-a432-b52a64767784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:03:05.653924 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-6e5ded4d-4aab-4c7b-a432-b52a64767784 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:03:05.655162 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6e5ded4d-4aab-4c7b-a432-b52a64767784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:03:05.749486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 734/1486] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:03:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:03:05.787604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645b295-9230-431d-8775-451dffb5c2b6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:03:05.789150 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1645b295-9230-431d-8775-451dffb5c2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:03:05.891126 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645b295-9230-431d-8775-451dffb5c2b6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:03:05.966040 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:03:05.967871 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb88c48-1257-42e9-9c39-6043b2cf4d0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:03:05.968558 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb88c48-1257-42e9-9c39-6043b2cf4d0e service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:03:05.969044 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb88c48-1257-42e9-9c39-6043b2cf4d0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:03:05.969470 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb88c48-1257-42e9-9c39-6043b2cf4d0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:03:05.971289 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-aeb88c48-1257-42e9-9c39-6043b2cf4d0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:03:06.077461 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb88c48-1257-42e9-9c39-6043b2cf4d0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:03:06.078885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb88c48-1257-42e9-9c39-6043b2cf4d0e service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:03:06.096230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 753/1487] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:03:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:03:08.208747 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ff77e-d881-4532-b00e-1812d7a8505a None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:03:08.209281 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ff77e-d881-4532-b00e-1812d7a8505a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:03:08.209281 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ff77e-d881-4532-b00e-1812d7a8505a None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:03:08.210603 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 735/1488] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:03:08.215835 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e2634-10ad-43ec-8a00-a5cc0a0600d2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:03:08.216335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e2634-10ad-43ec-8a00-a5cc0a0600d2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:03:08.216504 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e2634-10ad-43ec-8a00-a5cc0a0600d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:03:08.282555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ea4e2634-10ad-43ec-8a00-a5cc0a0600d2 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:03:08.283780 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ea4e2634-10ad-43ec-8a00-a5cc0a0600d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:03:08.337608 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:03:08.381031 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 754/1489] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:03:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:03:08.421481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ff77e-d881-4532-b00e-1812d7a8505a None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:03:08.423335 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c71ff77e-d881-4532-b00e-1812d7a8505a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:03:08.470000 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:03:08.540845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ff77e-d881-4532-b00e-1812d7a8505a None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:03:08.620023 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77dc7a-54c3-4453-9af6-6499a673faca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:03:08.620686 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77dc7a-54c3-4453-9af6-6499a673faca service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:03:08.620982 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77dc7a-54c3-4453-9af6-6499a673faca service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:03:08.621266 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77dc7a-54c3-4453-9af6-6499a673faca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:03:08.622917 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9f77dc7a-54c3-4453-9af6-6499a673faca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:03:08.702326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:03:08.731815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f77dc7a-54c3-4453-9af6-6499a673faca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:03:08.733224 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f77dc7a-54c3-4453-9af6-6499a673faca service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:03:08.751801 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 736/1490] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:03:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:03:10.944725 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbb5a9-c514-4def-83ac-04964d302d93 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:03:10.945249 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbb5a9-c514-4def-83ac-04964d302d93 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:03:10.945681 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbb5a9-c514-4def-83ac-04964d302d93 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:03:10.948311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 755/1491] 10.222.0.94 () {54 vars in 926 bytes} [Mon Jul 18 14:03:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:03:10.957031 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a1204-48fb-43a2-b606-a99af00e0d6c None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:03:10.957526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a1204-48fb-43a2-b606-a99af00e0d6c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:03:10.957633 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a1204-48fb-43a2-b606-a99af00e0d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:03:11.022526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-582a1204-48fb-43a2-b606-a99af00e0d6c None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:03:11.023741 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-582a1204-48fb-43a2-b606-a99af00e0d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:03:11.120591 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 737/1492] 10.222.0.94 () {58 vars in 1023 bytes} [Mon Jul 18 14:03:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:03:11.161103 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dbb5a9-c514-4def-83ac-04964d302d93 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:03:11.162702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c9dbb5a9-c514-4def-83ac-04964d302d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:03:11.266427 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dbb5a9-c514-4def-83ac-04964d302d93 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:03:11.341034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bcbe3d-918d-4241-bb4c-7d6cd7fd6fdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:03:11.341688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcbe3d-918d-4241-bb4c-7d6cd7fd6fdb service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:03:11.342000 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcbe3d-918d-4241-bb4c-7d6cd7fd6fdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:03:11.342278 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcbe3d-918d-4241-bb4c-7d6cd7fd6fdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:03:11.343902 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c8bcbe3d-918d-4241-bb4c-7d6cd7fd6fdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:03:11.352887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:03:11.449373 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bcbe3d-918d-4241-bb4c-7d6cd7fd6fdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:03:11.450798 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bcbe3d-918d-4241-bb4c-7d6cd7fd6fdb service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:03:11.468673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 756/1493] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:03:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:03:21.118182 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=82 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:03:21.466383 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=50 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:37.634112 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bbfae-ab3d-4399-9947-500c2aafa716 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:37.634681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bbfae-ab3d-4399-9947-500c2aafa716 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:37.634681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bbfae-ab3d-4399-9947-500c2aafa716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:37.713177 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-799bbfae-ab3d-4399-9947-500c2aafa716 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:37.714896 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-799bbfae-ab3d-4399-9947-500c2aafa716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:37.812105 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 738/1494] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:04:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:04:37.820710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363901ac-1ae9-41e6-934e-9edd0b5ffe70 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:37.821232 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363901ac-1ae9-41e6-934e-9edd0b5ffe70 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:37.821332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363901ac-1ae9-41e6-934e-9edd0b5ffe70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:37.894574 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-363901ac-1ae9-41e6-934e-9edd0b5ffe70 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:37.895703 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-363901ac-1ae9-41e6-934e-9edd0b5ffe70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:37.991299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 757/1495] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:04:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:04:37.996555 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bbfae-ab3d-4399-9947-500c2aafa716 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:37.997764 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-799bbfae-ab3d-4399-9947-500c2aafa716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:38.009209 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bbfae-ab3d-4399-9947-500c2aafa716 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:38.069298 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:38.107404 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e49a1-288e-4b24-bff2-fa3813d58bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:38.108621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e49a1-288e-4b24-bff2-fa3813d58bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:38.108944 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e49a1-288e-4b24-bff2-fa3813d58bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:38.109230 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e49a1-288e-4b24-bff2-fa3813d58bb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:38.109967 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e49a1-288e-4b24-bff2-fa3813d58bb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:38.110282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e49a1-288e-4b24-bff2-fa3813d58bb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:38.111297 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: WARNING py.warnings [None req-4f3e49a1-288e-4b24-bff2-fa3813d58bb1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:04:38.111297 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: warnings.warn(msg) Jul 18 14:04:38.111297 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: Jul 18 14:04:38.112230 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e49a1-288e-4b24-bff2-fa3813d58bb1 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:38.124280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 739/1496] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:38.132529 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363901ac-1ae9-41e6-934e-9edd0b5ffe70 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:38.141765 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363901ac-1ae9-41e6-934e-9edd0b5ffe70 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:38.202747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:38.237333 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db478d-c590-4bef-af66-cfe41f16a44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:38.237996 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db478d-c590-4bef-af66-cfe41f16a44a admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:38.238286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db478d-c590-4bef-af66-cfe41f16a44a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:38.238567 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db478d-c590-4bef-af66-cfe41f16a44a admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:38.239271 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db478d-c590-4bef-af66-cfe41f16a44a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:38.241028 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-10db478d-c590-4bef-af66-cfe41f16a44a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:04:38.241028 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Jul 18 14:04:38.241028 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: Jul 18 14:04:38.242231 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db478d-c590-4bef-af66-cfe41f16a44a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:38.297766 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 758/1497] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:04:38 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:04:38.301941 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e49a1-288e-4b24-bff2-fa3813d58bb1 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:38.303703 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4f3e49a1-288e-4b24-bff2-fa3813d58bb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:38.337631 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e49a1-288e-4b24-bff2-fa3813d58bb1 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:38.428203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e9fcf-9405-4a3b-841d-b49278950c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:38.428867 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e9fcf-9405-4a3b-841d-b49278950c25 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:38.429150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e9fcf-9405-4a3b-841d-b49278950c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:38.429424 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e9fcf-9405-4a3b-841d-b49278950c25 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:38.430117 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e9fcf-9405-4a3b-841d-b49278950c25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:38.431120 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: WARNING py.warnings [None req-e31e9fcf-9405-4a3b-841d-b49278950c25 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:04:38.431120 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: warnings.warn(msg) Jul 18 14:04:38.431120 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: Jul 18 14:04:38.432352 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e9fcf-9405-4a3b-841d-b49278950c25 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:38.472786 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 740/1498] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:04:38 2022] POST /identity/v3/users => generated 290 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:04:38.477712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db478d-c590-4bef-af66-cfe41f16a44a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:38.479232 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-10db478d-c590-4bef-af66-cfe41f16a44a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:38.543647 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db478d-c590-4bef-af66-cfe41f16a44a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:38.625128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:38.636931 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf3733-cf04-4627-a06f-7a0a8fdb9ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:38.637603 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf3733-cf04-4627-a06f-7a0a8fdb9ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:38.637892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf3733-cf04-4627-a06f-7a0a8fdb9ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:38.638173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf3733-cf04-4627-a06f-7a0a8fdb9ab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:38.638896 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cf3733-cf04-4627-a06f-7a0a8fdb9ab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:38.639909 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-99cf3733-cf04-4627-a06f-7a0a8fdb9ab7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:04:38.639909 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Jul 18 14:04:38.639909 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: Jul 18 14:04:38.640955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cf3733-cf04-4627-a06f-7a0a8fdb9ab7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:38.653305 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 759/1499] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:04:38 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:04:38.657563 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e9fcf-9405-4a3b-841d-b49278950c25 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:38.659078 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e31e9fcf-9405-4a3b-841d-b49278950c25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:38.671173 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e9fcf-9405-4a3b-841d-b49278950c25 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:38.696905 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:38.767022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75c50c-b0c1-4129-9ce6-954e2e7bcdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:38.767622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75c50c-b0c1-4129-9ce6-954e2e7bcdf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:38.767910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75c50c-b0c1-4129-9ce6-954e2e7bcdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:38.768225 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75c50c-b0c1-4129-9ce6-954e2e7bcdf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:38.768943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75c50c-b0c1-4129-9ce6-954e2e7bcdf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:38.769962 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: WARNING py.warnings [None req-0a75c50c-b0c1-4129-9ce6-954e2e7bcdf5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:04:38.769962 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: warnings.warn(msg) Jul 18 14:04:38.769962 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: Jul 18 14:04:38.770846 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75c50c-b0c1-4129-9ce6-954e2e7bcdf5 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:38.782988 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 741/1500] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:04:38 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:04:38.787203 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf3733-cf04-4627-a06f-7a0a8fdb9ab7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:38.788798 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-99cf3733-cf04-4627-a06f-7a0a8fdb9ab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:38.823011 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf3733-cf04-4627-a06f-7a0a8fdb9ab7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:38.916094 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514686a-64c8-4560-acd0-45fd2c780afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:38.917591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514686a-64c8-4560-acd0-45fd2c780afc admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:38.917885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514686a-64c8-4560-acd0-45fd2c780afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:38.918163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514686a-64c8-4560-acd0-45fd2c780afc admin admin] PATH_INFO: `/v3/projects/49f517a1fa284b60b19d44f1d250780b/users/f6843888dc1c42e6ba23d7508e4f7f95/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:38.939447 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514686a-64c8-4560-acd0-45fd2c780afc admin admin] RBAC: Authorizing `identity:create_grant(project_id=49f517a1fa284b60b19d44f1d250780b, user_id=f6843888dc1c42e6ba23d7508e4f7f95, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:38.940520 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-c514686a-64c8-4560-acd0-45fd2c780afc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:04:38.940520 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Jul 18 14:04:38.940520 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: Jul 18 14:04:38.942738 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514686a-64c8-4560-acd0-45fd2c780afc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:38.967836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 760/1501] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:04:38 2022] PUT /identity/v3/projects/49f517a1fa284b60b19d44f1d250780b/users/f6843888dc1c42e6ba23d7508e4f7f95/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:04:38.980497 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab62bc2-9d28-42ca-9a1d-4533ce9c9d7b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:38.980792 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab62bc2-9d28-42ca-9a1d-4533ce9c9d7b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:38.981073 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab62bc2-9d28-42ca-9a1d-4533ce9c9d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:39.036519 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-aab62bc2-9d28-42ca-9a1d-4533ce9c9d7b None None] MFA Rules not processed for user `f6843888dc1c42e6ba23d7508e4f7f95`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:39.037693 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-aab62bc2-9d28-42ca-9a1d-4533ce9c9d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:39.233481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 742/1502] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:04:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:04:39.247495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9aa31e-53e3-4316-840d-e4b104eb5ce2 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:39.247805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9aa31e-53e3-4316-840d-e4b104eb5ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:39.248108 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9aa31e-53e3-4316-840d-e4b104eb5ce2 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:39.249475 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 761/1503] 10.222.0.94 () {54 vars in 964 bytes} [Mon Jul 18 14:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:39.254258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98871233-784d-48a2-8ae3-61ca9e2ee92d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:39.254554 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98871233-784d-48a2-8ae3-61ca9e2ee92d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:39.254840 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98871233-784d-48a2-8ae3-61ca9e2ee92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:39.285329 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:39.332287 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-98871233-784d-48a2-8ae3-61ca9e2ee92d None None] MFA Rules not processed for user `45ee15f7716541a1a61443069d5cf4f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:39.333492 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-98871233-784d-48a2-8ae3-61ca9e2ee92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:39.409754 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 743/1504] 10.222.0.94 () {58 vars in 1061 bytes} [Mon Jul 18 14:04:39 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:04:39.417527 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9aa31e-53e3-4316-840d-e4b104eb5ce2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:39.419049 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6b9aa31e-53e3-4316-840d-e4b104eb5ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:39.469454 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:39.527120 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9aa31e-53e3-4316-840d-e4b104eb5ce2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:39.610549 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f45d7bf-48e1-4df3-9bac-b3bd119f24a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:39.611112 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45d7bf-48e1-4df3-9bac-b3bd119f24a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:39.611399 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45d7bf-48e1-4df3-9bac-b3bd119f24a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:39.611677 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45d7bf-48e1-4df3-9bac-b3bd119f24a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:39.613712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1f45d7bf-48e1-4df3-9bac-b3bd119f24a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:39.719909 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f45d7bf-48e1-4df3-9bac-b3bd119f24a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:39.721377 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f45d7bf-48e1-4df3-9bac-b3bd119f24a9 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:39.725705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:39.740753 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 762/1505] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:04:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:04:39.784226 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1c1f0-ff3c-41e6-ad55-ba8b40a7f204 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:39.784665 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1c1f0-ff3c-41e6-ad55-ba8b40a7f204 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:39.784793 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1c1f0-ff3c-41e6-ad55-ba8b40a7f204 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:39.786088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 744/1506] 10.222.0.94 () {52 vars in 829 bytes} [Mon Jul 18 14:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:39.790050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f45d7bf-48e1-4df3-9bac-b3bd119f24a9 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:39.791429 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1f45d7bf-48e1-4df3-9bac-b3bd119f24a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:39.903877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f45d7bf-48e1-4df3-9bac-b3bd119f24a9 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:40.004378 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f08c5-6c5a-46f2-8735-9f5da0f0ef17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:40.005258 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f08c5-6c5a-46f2-8735-9f5da0f0ef17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:40.005548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f08c5-6c5a-46f2-8735-9f5da0f0ef17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:40.005830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f08c5-6c5a-46f2-8735-9f5da0f0ef17 admin admin] PATH_INFO: `/v3/users/f6843888dc1c42e6ba23d7508e4f7f95` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:40.018558 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f08c5-6c5a-46f2-8735-9f5da0f0ef17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6843888dc1c42e6ba23d7508e4f7f95)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:40.019623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-6b0f08c5-6c5a-46f2-8735-9f5da0f0ef17 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:04:40.019623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Jul 18 14:04:40.019623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: Jul 18 14:04:40.020890 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f08c5-6c5a-46f2-8735-9f5da0f0ef17 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:40.030551 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:40.085433 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6b0f08c5-6c5a-46f2-8735-9f5da0f0ef17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6843888dc1c42e6ba23d7508e4f7f95'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:04:40.094917 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6b0f08c5-6c5a-46f2-8735-9f5da0f0ef17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6843888dc1c42e6ba23d7508e4f7f95'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:04:40.107784 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6b0f08c5-6c5a-46f2-8735-9f5da0f0ef17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6843888dc1c42e6ba23d7508e4f7f95'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:04:40.108268 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6b0f08c5-6c5a-46f2-8735-9f5da0f0ef17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6843888dc1c42e6ba23d7508e4f7f95'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:04:40.126496 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6b0f08c5-6c5a-46f2-8735-9f5da0f0ef17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6843888dc1c42e6ba23d7508e4f7f95'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:04:40.132119 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 763/1507] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:04:39 2022] DELETE /identity/v3/users/f6843888dc1c42e6ba23d7508e4f7f95 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:04:40.136658 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019e8d5-6fa4-4314-8680-9a8c8f925d10 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:40.136974 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019e8d5-6fa4-4314-8680-9a8c8f925d10 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:40.137259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019e8d5-6fa4-4314-8680-9a8c8f925d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:40.192787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b019e8d5-6fa4-4314-8680-9a8c8f925d10 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:40.193925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b019e8d5-6fa4-4314-8680-9a8c8f925d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:40.269549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:40.282620 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 745/1508] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:04:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:04:40.292887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f08c5-6c5a-46f2-8735-9f5da0f0ef17 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:40.294113 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6b0f08c5-6c5a-46f2-8735-9f5da0f0ef17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:40.395436 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f08c5-6c5a-46f2-8735-9f5da0f0ef17 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:40.449079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:40.469094 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5c4a8-e3f9-42de-90de-d882f3e45f72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:40.469769 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5c4a8-e3f9-42de-90de-d882f3e45f72 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:40.470057 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5c4a8-e3f9-42de-90de-d882f3e45f72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:40.470330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5c4a8-e3f9-42de-90de-d882f3e45f72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:40.471958 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c0e5c4a8-e3f9-42de-90de-d882f3e45f72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:40.588933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5c4a8-e3f9-42de-90de-d882f3e45f72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:40.590378 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5c4a8-e3f9-42de-90de-d882f3e45f72 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:40.608569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 764/1509] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:04:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:04:40.935937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019e8d5-6fa4-4314-8680-9a8c8f925d10 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:40.937300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b019e8d5-6fa4-4314-8680-9a8c8f925d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:40.951868 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019e8d5-6fa4-4314-8680-9a8c8f925d10 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:41.052530 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e527f01b-8527-41ab-958b-80856da26726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:41.053393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527f01b-8527-41ab-958b-80856da26726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:41.053675 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527f01b-8527-41ab-958b-80856da26726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:41.053945 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527f01b-8527-41ab-958b-80856da26726 admin admin] PATH_INFO: `/v3/projects/49f517a1fa284b60b19d44f1d250780b` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:41.065511 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e527f01b-8527-41ab-958b-80856da26726 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49f517a1fa284b60b19d44f1d250780b)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:41.066525 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: WARNING py.warnings [None req-e527f01b-8527-41ab-958b-80856da26726 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:04:41.066525 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: warnings.warn(msg) Jul 18 14:04:41.066525 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: Jul 18 14:04:41.067975 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e527f01b-8527-41ab-958b-80856da26726 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:41.123426 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e527f01b-8527-41ab-958b-80856da26726 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49f517a1fa284b60b19d44f1d250780b', 'request_id': 'req-e527f01b-8527-41ab-958b-80856da26726'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:04:41.129280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e527f01b-8527-41ab-958b-80856da26726 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49f517a1fa284b60b19d44f1d250780b', 'request_id': 'req-e527f01b-8527-41ab-958b-80856da26726'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:04:41.138733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e527f01b-8527-41ab-958b-80856da26726 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49f517a1fa284b60b19d44f1d250780b', 'request_id': 'req-e527f01b-8527-41ab-958b-80856da26726'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:04:41.169588 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e527f01b-8527-41ab-958b-80856da26726 admin admin] The token cache is being invalidate because project 49f517a1fa284b60b19d44f1d250780b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:04:41.169876 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e527f01b-8527-41ab-958b-80856da26726 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:04:41.170300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e527f01b-8527-41ab-958b-80856da26726 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:04:41.171736 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 746/1510] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:04:40 2022] DELETE /identity/v3/projects/49f517a1fa284b60b19d44f1d250780b => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:04:50.606233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=62 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:51.167628 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:25.660256 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a9630-f5ad-429e-8e9b-4d54b924e8c8 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:25.660821 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a9630-f5ad-429e-8e9b-4d54b924e8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:25.660821 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a9630-f5ad-429e-8e9b-4d54b924e8c8 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:25.662235 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 765/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:25.666919 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199bd81-adf1-4f6c-8bdd-a3ce35f7d231 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:25.667354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199bd81-adf1-4f6c-8bdd-a3ce35f7d231 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:25.667484 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199bd81-adf1-4f6c-8bdd-a3ce35f7d231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:25.746741 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-4199bd81-adf1-4f6c-8bdd-a3ce35f7d231 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:25.747960 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4199bd81-adf1-4f6c-8bdd-a3ce35f7d231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:25.849003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 747/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:06:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:06:26.001281 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a9630-f5ad-429e-8e9b-4d54b924e8c8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:26.003394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ef0a9630-f5ad-429e-8e9b-4d54b924e8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:26.120098 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a9630-f5ad-429e-8e9b-4d54b924e8c8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:26.126186 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8981f-ee53-4d92-9126-7f946162bea6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:26.126876 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8981f-ee53-4d92-9126-7f946162bea6 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:26.127181 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8981f-ee53-4d92-9126-7f946162bea6 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:26.127472 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8981f-ee53-4d92-9126-7f946162bea6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:26.135715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8981f-ee53-4d92-9126-7f946162bea6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:26.137508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8981f-ee53-4d92-9126-7f946162bea6 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:26.147832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 766/1513] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:06:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:06:26.966129 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6a35c-fc4c-4274-88f7-c65f15bcfd6e None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:26.966588 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6a35c-fc4c-4274-88f7-c65f15bcfd6e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:26.966695 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6a35c-fc4c-4274-88f7-c65f15bcfd6e None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:26.968606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 748/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:06:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:26.973595 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261a469-a313-47cd-aab8-e67ca7c2f5ba None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:26.974276 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261a469-a313-47cd-aab8-e67ca7c2f5ba None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:26.974541 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261a469-a313-47cd-aab8-e67ca7c2f5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:26.993207 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6261a469-a313-47cd-aab8-e67ca7c2f5ba None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:26.994412 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6261a469-a313-47cd-aab8-e67ca7c2f5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:27.002633 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 767/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:06:27.154470 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6a35c-fc4c-4274-88f7-c65f15bcfd6e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:27.169518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6a35c-fc4c-4274-88f7-c65f15bcfd6e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:27.174952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0344f55-2543-4b3e-b7da-2a900ab1784b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:27.175596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0344f55-2543-4b3e-b7da-2a900ab1784b service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:27.175884 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0344f55-2543-4b3e-b7da-2a900ab1784b service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:27.176196 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0344f55-2543-4b3e-b7da-2a900ab1784b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:27.184336 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0344f55-2543-4b3e-b7da-2a900ab1784b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:27.185905 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0344f55-2543-4b3e-b7da-2a900ab1784b service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:27.195442 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 749/1516] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:06:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:06:27.610638 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd2999-b12a-470c-9f42-4e3ccc7fb628 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:27.610875 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd2999-b12a-470c-9f42-4e3ccc7fb628 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:27.611130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd2999-b12a-470c-9f42-4e3ccc7fb628 None None] PATH_INFO: `/v3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:27.612488 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 768/1517] 10.222.0.87 () {54 vars in 925 bytes} [Mon Jul 18 14:06:27 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:27.619113 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7e2cf-9214-4f7f-92b9-23308ed308f1 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:27.619407 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7e2cf-9214-4f7f-92b9-23308ed308f1 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:27.619697 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7e2cf-9214-4f7f-92b9-23308ed308f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:27.678371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-bbd7e2cf-9214-4f7f-92b9-23308ed308f1 None None] MFA Rules not processed for user `dd5b2ba9e37041c78a99f1700a4970c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:27.679784 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bbd7e2cf-9214-4f7f-92b9-23308ed308f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:27.696584 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:27.751076 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 750/1518] 10.222.0.87 () {58 vars in 1015 bytes} [Mon Jul 18 14:06:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:06:36.974370 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd2f4d-96d7-4a7d-93dc-140289f095e9 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:36.974833 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd2f4d-96d7-4a7d-93dc-140289f095e9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:36.974946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd2f4d-96d7-4a7d-93dc-140289f095e9 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:36.976327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 769/1519] 10.222.0.94 () {54 vars in 961 bytes} [Mon Jul 18 14:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:36.981399 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f9df0-3fe6-4e1c-ab5d-80275012f151 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:36.981827 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f9df0-3fe6-4e1c-ab5d-80275012f151 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:36.981978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f9df0-3fe6-4e1c-ab5d-80275012f151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:36.988344 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=56 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:37.001100 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-710f9df0-3fe6-4e1c-ab5d-80275012f151 None None] MFA Rules not processed for user `dd5b2ba9e37041c78a99f1700a4970c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:37.002294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-710f9df0-3fe6-4e1c-ab5d-80275012f151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:37.010718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 751/1520] 10.222.0.94 () {58 vars in 1058 bytes} [Mon Jul 18 14:06:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:06:37.018106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd2f4d-96d7-4a7d-93dc-140289f095e9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:37.031737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd2f4d-96d7-4a7d-93dc-140289f095e9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:37.036675 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4fb00-a075-4756-ad17-28f2d45be0c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd5b2ba9e37041c78a99f1700a4970c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:37.037448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4fb00-a075-4756-ad17-28f2d45be0c8 service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:37.037747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4fb00-a075-4756-ad17-28f2d45be0c8 service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:37.038029 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4fb00-a075-4756-ad17-28f2d45be0c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:37.046264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c4fb00-a075-4756-ad17-28f2d45be0c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:37.048479 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c4fb00-a075-4756-ad17-28f2d45be0c8 service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:37.058089 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 770/1521] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:06:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:06:38.411350 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1d8b5-ce49-404c-bbcc-ef10116a7d7d None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:38.411891 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1d8b5-ce49-404c-bbcc-ef10116a7d7d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:38.411891 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1d8b5-ce49-404c-bbcc-ef10116a7d7d None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:38.413312 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 752/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:38.418099 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6209267-fd98-41b9-82c9-18aa5fcac568 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:38.418541 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6209267-fd98-41b9-82c9-18aa5fcac568 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:38.418700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6209267-fd98-41b9-82c9-18aa5fcac568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:38.437094 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c6209267-fd98-41b9-82c9-18aa5fcac568 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:38.438283 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c6209267-fd98-41b9-82c9-18aa5fcac568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:38.448729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 771/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:06:38.602692 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1d8b5-ce49-404c-bbcc-ef10116a7d7d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:38.617815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1d8b5-ce49-404c-bbcc-ef10116a7d7d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:38.623450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98035707-aeb2-4e05-b0f8-12266c700b3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:38.624130 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98035707-aeb2-4e05-b0f8-12266c700b3a service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:38.624416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98035707-aeb2-4e05-b0f8-12266c700b3a service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:38.624744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98035707-aeb2-4e05-b0f8-12266c700b3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:38.632821 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98035707-aeb2-4e05-b0f8-12266c700b3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:38.634389 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98035707-aeb2-4e05-b0f8-12266c700b3a service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:38.643662 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 753/1524] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:06:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:06:39.493426 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9beebf-08ea-41d3-93c2-710dcd7c80e4 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:39.493972 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9beebf-08ea-41d3-93c2-710dcd7c80e4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:39.493972 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9beebf-08ea-41d3-93c2-710dcd7c80e4 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:39.495375 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 772/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:39.500142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256cc01-61a4-4dd8-8571-ba9fd776a448 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:39.500578 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256cc01-61a4-4dd8-8571-ba9fd776a448 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:39.500707 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256cc01-61a4-4dd8-8571-ba9fd776a448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:39.518779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d256cc01-61a4-4dd8-8571-ba9fd776a448 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:39.519983 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d256cc01-61a4-4dd8-8571-ba9fd776a448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:39.527669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 754/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:06:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:06:39.677552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9beebf-08ea-41d3-93c2-710dcd7c80e4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:39.691635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9beebf-08ea-41d3-93c2-710dcd7c80e4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:39.697359 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7139679-9b4a-4986-a495-af273247c2c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:39.697985 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7139679-9b4a-4986-a495-af273247c2c6 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:39.698270 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7139679-9b4a-4986-a495-af273247c2c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:39.698547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7139679-9b4a-4986-a495-af273247c2c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:39.707021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7139679-9b4a-4986-a495-af273247c2c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:39.708670 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7139679-9b4a-4986-a495-af273247c2c6 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:39.717849 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 773/1527] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:06:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:06:43.636854 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f0e84-7124-45c2-9935-35ba59524cc7 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:43.637440 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f0e84-7124-45c2-9935-35ba59524cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:43.637440 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f0e84-7124-45c2-9935-35ba59524cc7 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:43.639128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 755/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:43.644884 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03144798-152b-473e-8a8e-ffd32ace5b7a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:43.645309 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03144798-152b-473e-8a8e-ffd32ace5b7a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:43.645422 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03144798-152b-473e-8a8e-ffd32ace5b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:43.663152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-03144798-152b-473e-8a8e-ffd32ace5b7a None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:43.664431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-03144798-152b-473e-8a8e-ffd32ace5b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:43.672574 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 774/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:06:43.710499 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f0e84-7124-45c2-9935-35ba59524cc7 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:43.712227 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-996f0e84-7124-45c2-9935-35ba59524cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:43.791285 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f0e84-7124-45c2-9935-35ba59524cc7 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:43.794814 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a4fcd9-98b7-4a6c-a54c-525b9445bafd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:43.795495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4fcd9-98b7-4a6c-a54c-525b9445bafd service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:43.795791 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4fcd9-98b7-4a6c-a54c-525b9445bafd service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:43.796120 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4fcd9-98b7-4a6c-a54c-525b9445bafd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:43.805144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a4fcd9-98b7-4a6c-a54c-525b9445bafd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:43.806583 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a4fcd9-98b7-4a6c-a54c-525b9445bafd service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:43.816944 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 756/1530] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:06:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:06:45.820835 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6422daa-c8af-4995-a14b-0b13be47d280 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:45.821399 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6422daa-c8af-4995-a14b-0b13be47d280 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:45.821506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6422daa-c8af-4995-a14b-0b13be47d280 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:45.822815 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 775/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:45.828681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857df01a-4c9d-402e-af5a-3c27cdcc0ef2 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:45.829077 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857df01a-4c9d-402e-af5a-3c27cdcc0ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:45.829239 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857df01a-4c9d-402e-af5a-3c27cdcc0ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:45.847652 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-857df01a-4c9d-402e-af5a-3c27cdcc0ef2 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:45.848866 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-857df01a-4c9d-402e-af5a-3c27cdcc0ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:45.856724 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 757/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:06:45.893046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6422daa-c8af-4995-a14b-0b13be47d280 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:45.894745 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b6422daa-c8af-4995-a14b-0b13be47d280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:45.907604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6422daa-c8af-4995-a14b-0b13be47d280 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:45.911154 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770be54-4a61-4df8-870e-1ab09f691e8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:45.911790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770be54-4a61-4df8-870e-1ab09f691e8d service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:45.912119 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770be54-4a61-4df8-870e-1ab09f691e8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:45.912405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770be54-4a61-4df8-870e-1ab09f691e8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:45.921333 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770be54-4a61-4df8-870e-1ab09f691e8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:45.922765 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770be54-4a61-4df8-870e-1ab09f691e8d service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:45.932616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 776/1533] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:06:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:06:49.383593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d079d2-5a94-4692-b4dd-01a06cdc9b8c None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:49.384122 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d079d2-5a94-4692-b4dd-01a06cdc9b8c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:49.384226 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d079d2-5a94-4692-b4dd-01a06cdc9b8c None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:49.385627 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 758/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:49.390828 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5abd8-bf58-420c-b134-66fdf11cc1e6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:49.391270 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5abd8-bf58-420c-b134-66fdf11cc1e6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:49.391383 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5abd8-bf58-420c-b134-66fdf11cc1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:49.409032 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c2d5abd8-bf58-420c-b134-66fdf11cc1e6 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:49.410237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c2d5abd8-bf58-420c-b134-66fdf11cc1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:49.418127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 777/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:06:49.723896 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d079d2-5a94-4692-b4dd-01a06cdc9b8c None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:49.725590 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-92d079d2-5a94-4692-b4dd-01a06cdc9b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:49.791733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:49.805391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d079d2-5a94-4692-b4dd-01a06cdc9b8c None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:49.808949 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae12ba-31db-4274-ba7e-329e76c49bd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:49.809606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae12ba-31db-4274-ba7e-329e76c49bd8 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:49.809897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae12ba-31db-4274-ba7e-329e76c49bd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:49.810180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae12ba-31db-4274-ba7e-329e76c49bd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:49.819965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae12ba-31db-4274-ba7e-329e76c49bd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:49.821426 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae12ba-31db-4274-ba7e-329e76c49bd8 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:49.831375 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 759/1536] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:06:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:06:51.852839 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15ec17-494d-4a2c-842c-df14abd0347d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:51.853395 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15ec17-494d-4a2c-842c-df14abd0347d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:51.853395 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15ec17-494d-4a2c-842c-df14abd0347d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:51.854836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 778/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:51.860221 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4c3b2-abc4-46c2-bfa0-d74f774aa819 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:51.860642 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4c3b2-abc4-46c2-bfa0-d74f774aa819 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:51.860790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4c3b2-abc4-46c2-bfa0-d74f774aa819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:51.879066 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c5d4c3b2-abc4-46c2-bfa0-d74f774aa819 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:51.880300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c5d4c3b2-abc4-46c2-bfa0-d74f774aa819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:51.888139 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 760/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:06:52.189095 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15ec17-494d-4a2c-842c-df14abd0347d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:52.190791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2a15ec17-494d-4a2c-842c-df14abd0347d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:52.204690 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15ec17-494d-4a2c-842c-df14abd0347d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:52.208229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd762c6-c6e5-4f7f-8da9-dfac03678955 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:52.208865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd762c6-c6e5-4f7f-8da9-dfac03678955 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:52.209157 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd762c6-c6e5-4f7f-8da9-dfac03678955 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:52.209441 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd762c6-c6e5-4f7f-8da9-dfac03678955 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:52.218314 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd762c6-c6e5-4f7f-8da9-dfac03678955 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:52.219749 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd762c6-c6e5-4f7f-8da9-dfac03678955 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:52.229355 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 779/1539] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:06:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:07:01.874404 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=11 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:02.225041 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=33 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:22.496159 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76cb49-681a-4e76-afaa-92cc8e2d6802 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:22.496895 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76cb49-681a-4e76-afaa-92cc8e2d6802 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:22.496895 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76cb49-681a-4e76-afaa-92cc8e2d6802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:22.534330 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-4d76cb49-681a-4e76-afaa-92cc8e2d6802 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:22.535575 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4d76cb49-681a-4e76-afaa-92cc8e2d6802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:22.547130 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 761/1540] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:08:22.555634 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73dead-b244-4ad4-9a52-8658d04404aa None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:22.556210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73dead-b244-4ad4-9a52-8658d04404aa None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:22.556210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73dead-b244-4ad4-9a52-8658d04404aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:22.584307 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6d73dead-b244-4ad4-9a52-8658d04404aa None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:22.585556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6d73dead-b244-4ad4-9a52-8658d04404aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:22.600127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 780/1541] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:22.604807 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76cb49-681a-4e76-afaa-92cc8e2d6802 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:22.617743 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76cb49-681a-4e76-afaa-92cc8e2d6802 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:22.625786 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67238f1-a3f9-4cdc-85d5-dea8b37766f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:22.626528 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67238f1-a3f9-4cdc-85d5-dea8b37766f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:22.626820 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67238f1-a3f9-4cdc-85d5-dea8b37766f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:22.627105 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67238f1-a3f9-4cdc-85d5-dea8b37766f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:22.627903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67238f1-a3f9-4cdc-85d5-dea8b37766f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:22.628244 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67238f1-a3f9-4cdc-85d5-dea8b37766f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:22.629955 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67238f1-a3f9-4cdc-85d5-dea8b37766f4 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:22.648579 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 762/1542] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:22.656478 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73dead-b244-4ad4-9a52-8658d04404aa None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:22.666405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73dead-b244-4ad4-9a52-8658d04404aa None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:22.676071 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd05309-c5c1-456c-87a0-e5cb67c06fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:22.677244 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd05309-c5c1-456c-87a0-e5cb67c06fb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:22.677791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd05309-c5c1-456c-87a0-e5cb67c06fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:22.678335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd05309-c5c1-456c-87a0-e5cb67c06fb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:22.679359 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd05309-c5c1-456c-87a0-e5cb67c06fb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:22.682043 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd05309-c5c1-456c-87a0-e5cb67c06fb8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:22.744640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 781/1543] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:08:22 2022] POST /identity/v3/projects => generated 379 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:08:22.749134 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67238f1-a3f9-4cdc-85d5-dea8b37766f4 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:22.760761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77c309-427c-4f8a-936c-514643d02fc1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:22.761691 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77c309-427c-4f8a-936c-514643d02fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:22.762226 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77c309-427c-4f8a-936c-514643d02fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:22.763991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67238f1-a3f9-4cdc-85d5-dea8b37766f4 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:22.798879 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3f77c309-427c-4f8a-936c-514643d02fc1 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:22.800838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3f77c309-427c-4f8a-936c-514643d02fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:22.865881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be20746-b00b-4bd6-97c9-8d8fd039feef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:22.867259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be20746-b00b-4bd6-97c9-8d8fd039feef admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:22.867553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be20746-b00b-4bd6-97c9-8d8fd039feef admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:22.867841 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be20746-b00b-4bd6-97c9-8d8fd039feef admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:22.872708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be20746-b00b-4bd6-97c9-8d8fd039feef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:22.876206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be20746-b00b-4bd6-97c9-8d8fd039feef admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:22.919009 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 763/1544] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:08:22 2022] POST /identity/v3/users => generated 290 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:22.920967 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e77fb-6e00-416b-abd9-fb9411dea2bf None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:22.921972 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e77fb-6e00-416b-abd9-fb9411dea2bf None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:22.922267 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e77fb-6e00-416b-abd9-fb9411dea2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:22.938685 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 782/1545] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:22.939607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77c309-427c-4f8a-936c-514643d02fc1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:22.969653 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77c309-427c-4f8a-936c-514643d02fc1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:22.979269 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-4e8e77fb-6e00-416b-abd9-fb9411dea2bf None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:22.982386 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4e8e77fb-6e00-416b-abd9-fb9411dea2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:22.991634 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 764/1546] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:22.992957 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b37cdb-6c4f-4726-86dd-de898358cf09 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:22.993254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b37cdb-6c4f-4726-86dd-de898358cf09 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:22.993381 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e6c1d-5e02-4c0e-bf79-316982345bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:22.993597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b37cdb-6c4f-4726-86dd-de898358cf09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:22.993958 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e6c1d-5e02-4c0e-bf79-316982345bff admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:22.994254 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e6c1d-5e02-4c0e-bf79-316982345bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:22.994818 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e6c1d-5e02-4c0e-bf79-316982345bff admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:22.995611 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e6c1d-5e02-4c0e-bf79-316982345bff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:22.997275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e6c1d-5e02-4c0e-bf79-316982345bff admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:23.013334 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-44b37cdb-6c4f-4726-86dd-de898358cf09 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:23.014466 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-44b37cdb-6c4f-4726-86dd-de898358cf09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:23.015007 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 783/1547] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:23.016946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defab3c-1f91-4fd9-98e2-eaeca54f492d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.017464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defab3c-1f91-4fd9-98e2-eaeca54f492d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.018062 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defab3c-1f91-4fd9-98e2-eaeca54f492d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.023520 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 765/1548] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:23.024903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fd257-9e9d-4180-99b5-cabaf7c7e8f6 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.025567 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fd257-9e9d-4180-99b5-cabaf7c7e8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.025853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fd257-9e9d-4180-99b5-cabaf7c7e8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.041835 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8defab3c-1f91-4fd9-98e2-eaeca54f492d None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:23.043114 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8defab3c-1f91-4fd9-98e2-eaeca54f492d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:23.050709 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-dc8fd257-9e9d-4180-99b5-cabaf7c7e8f6 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:23.052224 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-dc8fd257-9e9d-4180-99b5-cabaf7c7e8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:23.061725 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 784/1549] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:23.061725 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea4301-0c06-4169-84cc-2f5ed0663059 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.061725 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea4301-0c06-4169-84cc-2f5ed0663059 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.061725 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea4301-0c06-4169-84cc-2f5ed0663059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.062132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 766/1550] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:23.064667 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e157b-d0db-460d-b40d-570e616ceed8 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.064667 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e157b-d0db-460d-b40d-570e616ceed8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.064667 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e157b-d0db-460d-b40d-570e616ceed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.081040 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e0ea4301-0c06-4169-84cc-2f5ed0663059 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:23.082428 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e0ea4301-0c06-4169-84cc-2f5ed0663059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:23.088116 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-375e157b-d0db-460d-b40d-570e616ceed8 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:23.089321 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-375e157b-d0db-460d-b40d-570e616ceed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:23.091024 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 785/1551] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:23.091809 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea4301-0c06-4169-84cc-2f5ed0663059 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:23.097343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 767/1552] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:23.098185 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e157b-d0db-460d-b40d-570e616ceed8 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:23.101528 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea4301-0c06-4169-84cc-2f5ed0663059 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:23.107068 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05368249-6499-4aec-8f6c-831ba3a81c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:23.107762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05368249-6499-4aec-8f6c-831ba3a81c5a admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.108083 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05368249-6499-4aec-8f6c-831ba3a81c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.108294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e157b-d0db-460d-b40d-570e616ceed8 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:23.108408 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05368249-6499-4aec-8f6c-831ba3a81c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.109089 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05368249-6499-4aec-8f6c-831ba3a81c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:23.110720 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05368249-6499-4aec-8f6c-831ba3a81c5a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:23.112963 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c488c-90c3-4df3-96ba-e8d5a84a10cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:23.113616 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c488c-90c3-4df3-96ba-e8d5a84a10cc admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.113904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c488c-90c3-4df3-96ba-e8d5a84a10cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.114215 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c488c-90c3-4df3-96ba-e8d5a84a10cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.114952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c488c-90c3-4df3-96ba-e8d5a84a10cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:23.115262 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c488c-90c3-4df3-96ba-e8d5a84a10cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:23.116988 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c488c-90c3-4df3-96ba-e8d5a84a10cc admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:23.121536 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 786/1553] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:23.122741 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5265023-d1bd-442b-bdfe-547b5838efe9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.123061 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5265023-d1bd-442b-bdfe-547b5838efe9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.123341 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5265023-d1bd-442b-bdfe-547b5838efe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.134888 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 768/1554] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:23.134888 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8efd4d-8ef3-4123-98bb-0fad6413193d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.134888 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8efd4d-8ef3-4123-98bb-0fad6413193d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.134888 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8efd4d-8ef3-4123-98bb-0fad6413193d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.142791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c5265023-d1bd-442b-bdfe-547b5838efe9 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:23.144025 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c5265023-d1bd-442b-bdfe-547b5838efe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:23.151443 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-6d8efd4d-8ef3-4123-98bb-0fad6413193d None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:23.152586 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 787/1555] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:23.152718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6d8efd4d-8ef3-4123-98bb-0fad6413193d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:23.153755 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957d921-d4fc-41f8-b8d8-92e8c9717e19 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.154052 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957d921-d4fc-41f8-b8d8-92e8c9717e19 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.154340 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957d921-d4fc-41f8-b8d8-92e8c9717e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.160817 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 769/1556] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:23.162064 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a102d-5cab-4e3e-b18e-bcfc5a9e9587 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.162376 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a102d-5cab-4e3e-b18e-bcfc5a9e9587 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.162659 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a102d-5cab-4e3e-b18e-bcfc5a9e9587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.173752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3957d921-d4fc-41f8-b8d8-92e8c9717e19 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:23.174966 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3957d921-d4fc-41f8-b8d8-92e8c9717e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:23.179931 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-e31a102d-5cab-4e3e-b18e-bcfc5a9e9587 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:23.181185 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e31a102d-5cab-4e3e-b18e-bcfc5a9e9587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:23.183591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 788/1557] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:23.184388 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957d921-d4fc-41f8-b8d8-92e8c9717e19 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:23.189642 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 770/1558] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:23.190853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96261ea1-a8ef-4ac4-a14a-6a3619415ab4 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.191145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96261ea1-a8ef-4ac4-a14a-6a3619415ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.191425 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96261ea1-a8ef-4ac4-a14a-6a3619415ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.196225 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957d921-d4fc-41f8-b8d8-92e8c9717e19 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:23.201614 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd9581-0e4e-402d-8d56-f647592cac68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:23.202275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd9581-0e4e-402d-8d56-f647592cac68 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.202563 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd9581-0e4e-402d-8d56-f647592cac68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.202843 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd9581-0e4e-402d-8d56-f647592cac68 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.203585 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd9581-0e4e-402d-8d56-f647592cac68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:23.203899 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd9581-0e4e-402d-8d56-f647592cac68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:23.204948 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-c3bd9581-0e4e-402d-8d56-f647592cac68 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:08:23.204948 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Jul 18 14:08:23.204948 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: Jul 18 14:08:23.205842 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd9581-0e4e-402d-8d56-f647592cac68 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:23.216183 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-96261ea1-a8ef-4ac4-a14a-6a3619415ab4 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:23.217474 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-96261ea1-a8ef-4ac4-a14a-6a3619415ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:23.218104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 789/1559] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:23.218865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd9581-0e4e-402d-8d56-f647592cac68 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:23.227211 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 771/1560] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:23.228326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96261ea1-a8ef-4ac4-a14a-6a3619415ab4 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:23.228447 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd9581-0e4e-402d-8d56-f647592cac68 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:23.232945 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca88c1-c19b-488a-b440-31546ba382fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:23.233816 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca88c1-c19b-488a-b440-31546ba382fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.234103 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca88c1-c19b-488a-b440-31546ba382fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.234415 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca88c1-c19b-488a-b440-31546ba382fb admin admin] PATH_INFO: `/v3/projects/c3667a71bb064e738026bb620657d600/users/90d14aedc7b343e2996b7295b3733d2c/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.237918 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96261ea1-a8ef-4ac4-a14a-6a3619415ab4 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:23.242393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d06942-0d46-4657-bcd7-4010379727ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:23.243035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d06942-0d46-4657-bcd7-4010379727ec admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.243319 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d06942-0d46-4657-bcd7-4010379727ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.243595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d06942-0d46-4657-bcd7-4010379727ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.244310 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d06942-0d46-4657-bcd7-4010379727ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:23.245883 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: WARNING py.warnings [None req-31d06942-0d46-4657-bcd7-4010379727ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:08:23.245883 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: warnings.warn(msg) Jul 18 14:08:23.245883 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: Jul 18 14:08:23.247070 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d06942-0d46-4657-bcd7-4010379727ec admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:23.249740 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca88c1-c19b-488a-b440-31546ba382fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3667a71bb064e738026bb620657d600, user_id=90d14aedc7b343e2996b7295b3733d2c, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:23.252520 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca88c1-c19b-488a-b440-31546ba382fb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:23.267574 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 790/1561] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:08:23 2022] PUT /identity/v3/projects/c3667a71bb064e738026bb620657d600/users/90d14aedc7b343e2996b7295b3733d2c/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:23.269145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883f832-586c-46d0-8744-9faf55154656 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.269452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883f832-586c-46d0-8744-9faf55154656 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.269755 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883f832-586c-46d0-8744-9faf55154656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.274559 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 772/1562] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:23.275913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d06942-0d46-4657-bcd7-4010379727ec admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:23.288158 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c883f832-586c-46d0-8744-9faf55154656 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:23.289405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c883f832-586c-46d0-8744-9faf55154656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:23.324997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:23.329613 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:23.344387 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d06942-0d46-4657-bcd7-4010379727ec admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:23.349051 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f428097-9091-4528-b53a-d12f79388108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:23.349821 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f428097-9091-4528-b53a-d12f79388108 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.350347 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f428097-9091-4528-b53a-d12f79388108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.350747 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f428097-9091-4528-b53a-d12f79388108 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.351583 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f428097-9091-4528-b53a-d12f79388108 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:23.352035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f428097-9091-4528-b53a-d12f79388108 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:23.352205 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 791/1563] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:23.352879 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883f832-586c-46d0-8744-9faf55154656 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:23.357382 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f428097-9091-4528-b53a-d12f79388108 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:23.362049 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883f832-586c-46d0-8744-9faf55154656 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:23.366806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b6fae-153b-451c-8067-f01b43cd2ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:23.367448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b6fae-153b-451c-8067-f01b43cd2ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.367740 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b6fae-153b-451c-8067-f01b43cd2ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.368063 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b6fae-153b-451c-8067-f01b43cd2ec8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.368795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b6fae-153b-451c-8067-f01b43cd2ec8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:23.368997 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 773/1564] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:23.369099 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b6fae-153b-451c-8067-f01b43cd2ec8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:23.370769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4ced5-5573-4750-9d4a-967e5d29e8d5 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.371061 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4ced5-5573-4750-9d4a-967e5d29e8d5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.371304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b6fae-153b-451c-8067-f01b43cd2ec8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:23.371413 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4ced5-5573-4750-9d4a-967e5d29e8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.387191 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 792/1565] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:23.388014 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b6fae-153b-451c-8067-f01b43cd2ec8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:23.393753 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-31c4ced5-5573-4750-9d4a-967e5d29e8d5 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:23.395292 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-31c4ced5-5573-4750-9d4a-967e5d29e8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:23.401664 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b6fae-153b-451c-8067-f01b43cd2ec8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:23.403232 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 774/1566] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:23.404094 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4ced5-5573-4750-9d4a-967e5d29e8d5 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:23.411686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b157eab-e74b-4232-86d4-419be8e4ada5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:23.412384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b157eab-e74b-4232-86d4-419be8e4ada5 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.412705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b157eab-e74b-4232-86d4-419be8e4ada5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.412986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b157eab-e74b-4232-86d4-419be8e4ada5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.413644 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4ced5-5573-4750-9d4a-967e5d29e8d5 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:23.413751 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b157eab-e74b-4232-86d4-419be8e4ada5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:23.415817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b157eab-e74b-4232-86d4-419be8e4ada5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:23.419708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b785af46-64e7-41a1-8a42-02f49efba47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:23.420057 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785af46-64e7-41a1-8a42-02f49efba47c admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.420329 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785af46-64e7-41a1-8a42-02f49efba47c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.420615 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785af46-64e7-41a1-8a42-02f49efba47c admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.421344 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b785af46-64e7-41a1-8a42-02f49efba47c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:23.421653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b785af46-64e7-41a1-8a42-02f49efba47c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:23.423243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b785af46-64e7-41a1-8a42-02f49efba47c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:23.435786 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 775/1567] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:23.436635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b785af46-64e7-41a1-8a42-02f49efba47c admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:23.443815 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 793/1568] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/projects => generated 391 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:23.445226 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ed734-98f5-4a99-b22d-e5e8ae91769c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.445524 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ed734-98f5-4a99-b22d-e5e8ae91769c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.445808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08ed734-98f5-4a99-b22d-e5e8ae91769c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.453489 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b785af46-64e7-41a1-8a42-02f49efba47c admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:23.472780 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b08ed734-98f5-4a99-b22d-e5e8ae91769c None None] MFA Rules not processed for user `90d14aedc7b343e2996b7295b3733d2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:23.474326 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b08ed734-98f5-4a99-b22d-e5e8ae91769c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:23.513084 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b569a-dd85-46ba-bc57-7b691d146501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:23.513762 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b569a-dd85-46ba-bc57-7b691d146501 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.514166 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b569a-dd85-46ba-bc57-7b691d146501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.514451 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b569a-dd85-46ba-bc57-7b691d146501 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.515170 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b569a-dd85-46ba-bc57-7b691d146501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:23.517171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b569a-dd85-46ba-bc57-7b691d146501 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:23.539620 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 776/1569] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:23.540485 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b569a-dd85-46ba-bc57-7b691d146501 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:23.545093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 794/1570] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:23.545093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ed734-98f5-4a99-b22d-e5e8ae91769c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:23.550090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b569a-dd85-46ba-bc57-7b691d146501 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:23.554922 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08ed734-98f5-4a99-b22d-e5e8ae91769c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:23.555296 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b8213-0f92-44c0-b144-e724ff975871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:23.555987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b8213-0f92-44c0-b144-e724ff975871 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.556263 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b8213-0f92-44c0-b144-e724ff975871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.556545 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b8213-0f92-44c0-b144-e724ff975871 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.557267 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b8213-0f92-44c0-b144-e724ff975871 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:23.557613 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b8213-0f92-44c0-b144-e724ff975871 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:23.559197 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b8213-0f92-44c0-b144-e724ff975871 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:23.559978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75624451-2596-45de-a5a9-8d481d3bf670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:23.560678 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75624451-2596-45de-a5a9-8d481d3bf670 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.560969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75624451-2596-45de-a5a9-8d481d3bf670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.561253 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75624451-2596-45de-a5a9-8d481d3bf670 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.561953 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75624451-2596-45de-a5a9-8d481d3bf670 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:23.564010 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75624451-2596-45de-a5a9-8d481d3bf670 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:23.579387 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 777/1571] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:23.579387 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b8213-0f92-44c0-b144-e724ff975871 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:23.589587 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b8213-0f92-44c0-b144-e724ff975871 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:23.594505 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d1780-3086-4cb1-817f-7eb451d83e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:23.595158 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d1780-3086-4cb1-817f-7eb451d83e00 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.595450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d1780-3086-4cb1-817f-7eb451d83e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.595732 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d1780-3086-4cb1-817f-7eb451d83e00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.596455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d1780-3086-4cb1-817f-7eb451d83e00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:23.601828 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 795/1572] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:23.601994 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d1780-3086-4cb1-817f-7eb451d83e00 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:23.602104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75624451-2596-45de-a5a9-8d481d3bf670 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:23.625649 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 778/1573] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:23.626531 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d1780-3086-4cb1-817f-7eb451d83e00 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:23.664976 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75624451-2596-45de-a5a9-8d481d3bf670 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:23.677635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:23.687852 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d1780-3086-4cb1-817f-7eb451d83e00 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:23.692903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfd47d-961d-441b-84bd-b98afb924470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:23.693575 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfd47d-961d-441b-84bd-b98afb924470 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.693865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfd47d-961d-441b-84bd-b98afb924470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.694207 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acfd47d-961d-441b-84bd-b98afb924470 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.694913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acfd47d-961d-441b-84bd-b98afb924470 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:23.696936 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acfd47d-961d-441b-84bd-b98afb924470 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:23.723317 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f220fec5-d30f-4f3a-89a3-dae595a9157f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:23.723518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 779/1574] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:23.724067 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220fec5-d30f-4f3a-89a3-dae595a9157f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.724247 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfd47d-961d-441b-84bd-b98afb924470 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:23.724360 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220fec5-d30f-4f3a-89a3-dae595a9157f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.724620 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220fec5-d30f-4f3a-89a3-dae595a9157f admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.725345 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f220fec5-d30f-4f3a-89a3-dae595a9157f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:23.725654 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f220fec5-d30f-4f3a-89a3-dae595a9157f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:23.727312 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f220fec5-d30f-4f3a-89a3-dae595a9157f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:23.731990 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:23.747151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 796/1575] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:23.747151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f220fec5-d30f-4f3a-89a3-dae595a9157f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:23.752241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f220fec5-d30f-4f3a-89a3-dae595a9157f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:23.793837 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acfd47d-961d-441b-84bd-b98afb924470 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:23.798972 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff85727-58a9-4e8c-a2af-be5e51f4fc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:23.799700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff85727-58a9-4e8c-a2af-be5e51f4fc9e admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.800044 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff85727-58a9-4e8c-a2af-be5e51f4fc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.800461 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff85727-58a9-4e8c-a2af-be5e51f4fc9e admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.801180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff85727-58a9-4e8c-a2af-be5e51f4fc9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:23.803171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff85727-58a9-4e8c-a2af-be5e51f4fc9e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:23.809056 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad8982-e4e5-47e6-88a5-5d70180a7408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:23.809742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad8982-e4e5-47e6-88a5-5d70180a7408 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.810025 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad8982-e4e5-47e6-88a5-5d70180a7408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.810300 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad8982-e4e5-47e6-88a5-5d70180a7408 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.811026 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad8982-e4e5-47e6-88a5-5d70180a7408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:23.812675 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad8982-e4e5-47e6-88a5-5d70180a7408 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:23.824654 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 797/1576] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:23.825446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad8982-e4e5-47e6-88a5-5d70180a7408 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:23.832905 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 780/1577] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:23.832905 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff85727-58a9-4e8c-a2af-be5e51f4fc9e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:23.835076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad8982-e4e5-47e6-88a5-5d70180a7408 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:23.840479 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93180adc-e91b-4ad6-8e88-55240bf894fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:23.841129 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93180adc-e91b-4ad6-8e88-55240bf894fc admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.841422 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93180adc-e91b-4ad6-8e88-55240bf894fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.841706 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93180adc-e91b-4ad6-8e88-55240bf894fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.842398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93180adc-e91b-4ad6-8e88-55240bf894fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:23.842910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff85727-58a9-4e8c-a2af-be5e51f4fc9e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:23.844347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93180adc-e91b-4ad6-8e88-55240bf894fc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:23.847547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60820e4-4872-4f80-bfaa-0f211a7c73cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:23.848215 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60820e4-4872-4f80-bfaa-0f211a7c73cb admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.848512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60820e4-4872-4f80-bfaa-0f211a7c73cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.848792 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60820e4-4872-4f80-bfaa-0f211a7c73cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.849475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60820e4-4872-4f80-bfaa-0f211a7c73cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:23.851355 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60820e4-4872-4f80-bfaa-0f211a7c73cb admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:23.873327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 798/1578] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:23.874704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93180adc-e91b-4ad6-8e88-55240bf894fc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:23.880350 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 781/1579] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:23.881164 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60820e4-4872-4f80-bfaa-0f211a7c73cb admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:23.943476 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60820e4-4872-4f80-bfaa-0f211a7c73cb admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:23.946378 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93180adc-e91b-4ad6-8e88-55240bf894fc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:23.949515 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c65069-cf70-4588-a88f-9c3bdb5831fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:23.950136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c65069-cf70-4588-a88f-9c3bdb5831fa admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.950422 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c65069-cf70-4588-a88f-9c3bdb5831fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.950705 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c65069-cf70-4588-a88f-9c3bdb5831fa admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.951395 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c65069-cf70-4588-a88f-9c3bdb5831fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:23.951714 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0181b-e5b7-4d67-8659-49537df19c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:23.952409 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0181b-e5b7-4d67-8659-49537df19c9a admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.952697 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0181b-e5b7-4d67-8659-49537df19c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.952977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0181b-e5b7-4d67-8659-49537df19c9a admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.953353 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c65069-cf70-4588-a88f-9c3bdb5831fa admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:23.953806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0181b-e5b7-4d67-8659-49537df19c9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:23.954845 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-57a0181b-e5b7-4d67-8659-49537df19c9a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:08:23.954845 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Jul 18 14:08:23.954845 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: Jul 18 14:08:23.956164 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0181b-e5b7-4d67-8659-49537df19c9a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:23.977607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 799/1580] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:23.978625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0181b-e5b7-4d67-8659-49537df19c9a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:23.979247 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 782/1581] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:23.980390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c65069-cf70-4588-a88f-9c3bdb5831fa admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:23.988665 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0181b-e5b7-4d67-8659-49537df19c9a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:23.989843 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c65069-cf70-4588-a88f-9c3bdb5831fa admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:23.993214 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd250c9-8532-460a-9b85-bb446a97985d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:23.993859 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd250c9-8532-460a-9b85-bb446a97985d admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.994142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd250c9-8532-460a-9b85-bb446a97985d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.994416 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd250c9-8532-460a-9b85-bb446a97985d admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.994596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc3fc7-57a8-418e-a364-d0a48b5237a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:23.995636 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd250c9-8532-460a-9b85-bb446a97985d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:23.995839 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc3fc7-57a8-418e-a364-d0a48b5237a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:23.996164 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc3fc7-57a8-418e-a364-d0a48b5237a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:23.996439 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fc3fc7-57a8-418e-a364-d0a48b5237a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:23.997116 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc3fc7-57a8-418e-a364-d0a48b5237a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:23.997581 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd250c9-8532-460a-9b85-bb446a97985d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:23.998977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fc3fc7-57a8-418e-a364-d0a48b5237a2 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.012371 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:24.018531 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 800/1582] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/users => generated 287 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:24.019712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd250c9-8532-460a-9b85-bb446a97985d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.026907 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 783/1583] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:08:23 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:24.028973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc3fc7-57a8-418e-a364-d0a48b5237a2 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.030525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd250c9-8532-460a-9b85-bb446a97985d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.069544 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:24.086766 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13f49d-1866-4a8c-b741-d57a0d907726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.087445 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13f49d-1866-4a8c-b741-d57a0d907726 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.087733 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13f49d-1866-4a8c-b741-d57a0d907726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.088047 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13f49d-1866-4a8c-b741-d57a0d907726 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.089313 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13f49d-1866-4a8c-b741-d57a0d907726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.090981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13f49d-1866-4a8c-b741-d57a0d907726 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.091295 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fc3fc7-57a8-418e-a364-d0a48b5237a2 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.095991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b6214-5321-4ca3-94f8-a717e7e975ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.096657 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b6214-5321-4ca3-94f8-a717e7e975ca admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.096940 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b6214-5321-4ca3-94f8-a717e7e975ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.097218 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b6214-5321-4ca3-94f8-a717e7e975ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.098082 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b6214-5321-4ca3-94f8-a717e7e975ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.099685 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b6214-5321-4ca3-94f8-a717e7e975ca admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.100851 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 801/1584] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:24.105500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13f49d-1866-4a8c-b741-d57a0d907726 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.113445 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 784/1585] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:24.113445 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b6214-5321-4ca3-94f8-a717e7e975ca admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.120481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13f49d-1866-4a8c-b741-d57a0d907726 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.121243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b6214-5321-4ca3-94f8-a717e7e975ca admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.125105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62890215-d46a-4182-bd94-64bf105443aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.125767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62890215-d46a-4182-bd94-64bf105443aa admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.126057 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62890215-d46a-4182-bd94-64bf105443aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.126339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62890215-d46a-4182-bd94-64bf105443aa admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.126436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579fa9e-5dc1-4c4a-8f3a-cd15a2a7a7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.127003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579fa9e-5dc1-4c4a-8f3a-cd15a2a7a7aa admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.127117 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62890215-d46a-4182-bd94-64bf105443aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.127291 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579fa9e-5dc1-4c4a-8f3a-cd15a2a7a7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.127569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579fa9e-5dc1-4c4a-8f3a-cd15a2a7a7aa admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.128295 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579fa9e-5dc1-4c4a-8f3a-cd15a2a7a7aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.129001 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62890215-d46a-4182-bd94-64bf105443aa admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.130166 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579fa9e-5dc1-4c4a-8f3a-cd15a2a7a7aa admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.152310 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 802/1586] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:08:24 2022] POST /identity/v3/users => generated 297 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:24.152729 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 785/1587] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:08:24 2022] POST /identity/v3/users => generated 289 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:24.153675 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62890215-d46a-4182-bd94-64bf105443aa admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.156608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579fa9e-5dc1-4c4a-8f3a-cd15a2a7a7aa admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.165319 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62890215-d46a-4182-bd94-64bf105443aa admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.166426 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579fa9e-5dc1-4c4a-8f3a-cd15a2a7a7aa admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.170090 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7596fbc-32ad-41c2-bcdd-c0dfdcf0813f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.170773 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7596fbc-32ad-41c2-bcdd-c0dfdcf0813f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.171064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7596fbc-32ad-41c2-bcdd-c0dfdcf0813f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.171344 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7596fbc-32ad-41c2-bcdd-c0dfdcf0813f admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.172132 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7596fbc-32ad-41c2-bcdd-c0dfdcf0813f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.173148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa706265-8f9c-48c1-b6cf-c98424e24564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.173785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7596fbc-32ad-41c2-bcdd-c0dfdcf0813f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.174293 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa706265-8f9c-48c1-b6cf-c98424e24564 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.174798 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa706265-8f9c-48c1-b6cf-c98424e24564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.175303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa706265-8f9c-48c1-b6cf-c98424e24564 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.176675 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa706265-8f9c-48c1-b6cf-c98424e24564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.179378 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa706265-8f9c-48c1-b6cf-c98424e24564 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.185315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 803/1588] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:24.185960 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7596fbc-32ad-41c2-bcdd-c0dfdcf0813f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.194035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 786/1589] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:24.194947 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa706265-8f9c-48c1-b6cf-c98424e24564 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.196265 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7596fbc-32ad-41c2-bcdd-c0dfdcf0813f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.202321 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a69bda-8df1-4c37-a1a3-0cc3621b019d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.202981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a69bda-8df1-4c37-a1a3-0cc3621b019d admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.203267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a69bda-8df1-4c37-a1a3-0cc3621b019d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.203548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a69bda-8df1-4c37-a1a3-0cc3621b019d admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.204300 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a69bda-8df1-4c37-a1a3-0cc3621b019d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.205347 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa706265-8f9c-48c1-b6cf-c98424e24564 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.205917 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a69bda-8df1-4c37-a1a3-0cc3621b019d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.210110 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1464d0-286e-4bc4-9a9f-0fd83740faca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.210785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1464d0-286e-4bc4-9a9f-0fd83740faca admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.211082 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1464d0-286e-4bc4-9a9f-0fd83740faca admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.211370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1464d0-286e-4bc4-9a9f-0fd83740faca admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.212087 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1464d0-286e-4bc4-9a9f-0fd83740faca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.214648 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1464d0-286e-4bc4-9a9f-0fd83740faca admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.216817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 804/1590] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:24.217707 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a69bda-8df1-4c37-a1a3-0cc3621b019d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.227343 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a69bda-8df1-4c37-a1a3-0cc3621b019d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.232391 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce571c6-7424-4d6b-864b-54750c9e3a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.233258 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce571c6-7424-4d6b-864b-54750c9e3a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.233550 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce571c6-7424-4d6b-864b-54750c9e3a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.233834 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce571c6-7424-4d6b-864b-54750c9e3a0b admin admin] PATH_INFO: `/v3/projects/26e4140751cf40c2adbfd92a29afd555/users/0542cd47ba77454ba25170292ed9aeb9/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.238067 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 787/1591] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:08:24 2022] POST /identity/v3/users => generated 290 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:24.238946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1464d0-286e-4bc4-9a9f-0fd83740faca admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.249274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce571c6-7424-4d6b-864b-54750c9e3a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=26e4140751cf40c2adbfd92a29afd555, user_id=0542cd47ba77454ba25170292ed9aeb9, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.249916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1464d0-286e-4bc4-9a9f-0fd83740faca admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.252061 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce571c6-7424-4d6b-864b-54750c9e3a0b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.254510 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5287fca-de9a-47df-84a7-a298b433209a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.255152 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5287fca-de9a-47df-84a7-a298b433209a admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.255434 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5287fca-de9a-47df-84a7-a298b433209a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.255709 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5287fca-de9a-47df-84a7-a298b433209a admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.256557 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5287fca-de9a-47df-84a7-a298b433209a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.258094 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5287fca-de9a-47df-84a7-a298b433209a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.266276 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 805/1592] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:08:24 2022] PUT /identity/v3/projects/26e4140751cf40c2adbfd92a29afd555/users/0542cd47ba77454ba25170292ed9aeb9/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:24.266885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce571c6-7424-4d6b-864b-54750c9e3a0b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.268723 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 788/1593] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:24.269522 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5287fca-de9a-47df-84a7-a298b433209a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.279842 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5287fca-de9a-47df-84a7-a298b433209a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.333998 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce571c6-7424-4d6b-864b-54750c9e3a0b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.336992 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ae01f-3641-41c9-9902-9233137c6584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.337707 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ae01f-3641-41c9-9902-9233137c6584 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.338020 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ae01f-3641-41c9-9902-9233137c6584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.338397 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ae01f-3641-41c9-9902-9233137c6584 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.338739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712eaef5-a2c9-4ffe-9d48-096b5d61d182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.339688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ae01f-3641-41c9-9902-9233137c6584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.340066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712eaef5-a2c9-4ffe-9d48-096b5d61d182 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.340360 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712eaef5-a2c9-4ffe-9d48-096b5d61d182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.340643 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712eaef5-a2c9-4ffe-9d48-096b5d61d182 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.341333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ae01f-3641-41c9-9902-9233137c6584 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.341445 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712eaef5-a2c9-4ffe-9d48-096b5d61d182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.342984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712eaef5-a2c9-4ffe-9d48-096b5d61d182 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.351364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 789/1594] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:24.352236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ae01f-3641-41c9-9902-9233137c6584 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.356508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 806/1595] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:24.357410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712eaef5-a2c9-4ffe-9d48-096b5d61d182 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.366520 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ae01f-3641-41c9-9902-9233137c6584 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.366807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712eaef5-a2c9-4ffe-9d48-096b5d61d182 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.371072 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bd4df-1133-4bea-80d5-641fb42213e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.371411 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e028c5-0fdc-4796-95f0-e965e7dbd193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.371716 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bd4df-1133-4bea-80d5-641fb42213e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.372033 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bd4df-1133-4bea-80d5-641fb42213e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.372139 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e028c5-0fdc-4796-95f0-e965e7dbd193 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.372316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bd4df-1133-4bea-80d5-641fb42213e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.373003 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e028c5-0fdc-4796-95f0-e965e7dbd193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.373114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641bd4df-1133-4bea-80d5-641fb42213e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.373292 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e028c5-0fdc-4796-95f0-e965e7dbd193 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.374007 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e028c5-0fdc-4796-95f0-e965e7dbd193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.374630 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641bd4df-1133-4bea-80d5-641fb42213e9 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.375585 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e028c5-0fdc-4796-95f0-e965e7dbd193 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.386171 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 807/1596] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:24.387678 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e028c5-0fdc-4796-95f0-e965e7dbd193 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.397287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e028c5-0fdc-4796-95f0-e965e7dbd193 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.402612 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 790/1597] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:24.402612 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bd4df-1133-4bea-80d5-641fb42213e9 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.402848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e7882-6406-46c5-955f-363bbc79fc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.403130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e7882-6406-46c5-955f-363bbc79fc92 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.403417 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e7882-6406-46c5-955f-363bbc79fc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.403725 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e7882-6406-46c5-955f-363bbc79fc92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.404521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e7882-6406-46c5-955f-363bbc79fc92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.406114 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e7882-6406-46c5-955f-363bbc79fc92 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.410649 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bd4df-1133-4bea-80d5-641fb42213e9 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.415264 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e189a-0f16-4335-aa9c-697bdf8740dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.415912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e189a-0f16-4335-aa9c-697bdf8740dc admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.416236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e189a-0f16-4335-aa9c-697bdf8740dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.416523 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e189a-0f16-4335-aa9c-697bdf8740dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.419756 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 808/1598] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:24.419901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e189a-0f16-4335-aa9c-697bdf8740dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.420048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e7882-6406-46c5-955f-363bbc79fc92 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.420142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e189a-0f16-4335-aa9c-697bdf8740dc admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.423332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:24.427213 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e7882-6406-46c5-955f-363bbc79fc92 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.430300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 791/1599] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:24.431514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f05ed-9781-4527-9b45-cf6ac66110b2 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.431809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f05ed-9781-4527-9b45-cf6ac66110b2 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.432118 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f05ed-9781-4527-9b45-cf6ac66110b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.435259 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab20ee-5707-41dc-8f94-aa2266d8df0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.436158 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab20ee-5707-41dc-8f94-aa2266d8df0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.436446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab20ee-5707-41dc-8f94-aa2266d8df0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.436726 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab20ee-5707-41dc-8f94-aa2266d8df0b admin admin] PATH_INFO: `/v3/projects/eaf9f75abc034491bd1b8846c05bf625/users/3720c2225b51464abee34d05a69b9db7/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.452345 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-135f05ed-9781-4527-9b45-cf6ac66110b2 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:24.452546 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab20ee-5707-41dc-8f94-aa2266d8df0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaf9f75abc034491bd1b8846c05bf625, user_id=3720c2225b51464abee34d05a69b9db7, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.453554 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-135f05ed-9781-4527-9b45-cf6ac66110b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:24.455089 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab20ee-5707-41dc-8f94-aa2266d8df0b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.461658 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 792/1600] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:24.465752 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135f05ed-9781-4527-9b45-cf6ac66110b2 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.468226 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 809/1601] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:08:24 2022] PUT /identity/v3/projects/eaf9f75abc034491bd1b8846c05bf625/users/3720c2225b51464abee34d05a69b9db7/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:24.469034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab20ee-5707-41dc-8f94-aa2266d8df0b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.478707 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135f05ed-9781-4527-9b45-cf6ac66110b2 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.520576 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:24.533269 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab20ee-5707-41dc-8f94-aa2266d8df0b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.536763 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438cb0c-e474-41ca-b41b-a3701855a003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.537423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438cb0c-e474-41ca-b41b-a3701855a003 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.537713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438cb0c-e474-41ca-b41b-a3701855a003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.538002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438cb0c-e474-41ca-b41b-a3701855a003 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.538651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb9162-8a8b-417c-85cb-62707a7dc890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.538779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3438cb0c-e474-41ca-b41b-a3701855a003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.539751 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb9162-8a8b-417c-85cb-62707a7dc890 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.540164 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb9162-8a8b-417c-85cb-62707a7dc890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.540357 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3438cb0c-e474-41ca-b41b-a3701855a003 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.540641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb9162-8a8b-417c-85cb-62707a7dc890 admin admin] PATH_INFO: `/v3/projects/c70edb2715d64ceaab313111a78ed840/users/d9fbda50766d4ca98dafeedbfa2c36d2/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.550225 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 793/1602] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:24.551012 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438cb0c-e474-41ca-b41b-a3701855a003 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.557364 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb9162-8a8b-417c-85cb-62707a7dc890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c70edb2715d64ceaab313111a78ed840, user_id=d9fbda50766d4ca98dafeedbfa2c36d2, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.560954 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeb9162-8a8b-417c-85cb-62707a7dc890 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.561538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438cb0c-e474-41ca-b41b-a3701855a003 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.566870 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce41c9-09e6-4976-97c5-e2a16555c8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.568152 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce41c9-09e6-4976-97c5-e2a16555c8e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.568483 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce41c9-09e6-4976-97c5-e2a16555c8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.568769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce41c9-09e6-4976-97c5-e2a16555c8e4 admin admin] PATH_INFO: `/v3/projects/1a0691238b54478d81de9ada168aea28/users/81cd7a7f6a9c4dbbaa3548cbca297762/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.574250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 810/1603] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:08:24 2022] PUT /identity/v3/projects/c70edb2715d64ceaab313111a78ed840/users/d9fbda50766d4ca98dafeedbfa2c36d2/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:24.575096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb9162-8a8b-417c-85cb-62707a7dc890 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.584298 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce41c9-09e6-4976-97c5-e2a16555c8e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a0691238b54478d81de9ada168aea28, user_id=81cd7a7f6a9c4dbbaa3548cbca297762, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.585314 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: WARNING py.warnings [None req-a8ce41c9-09e6-4976-97c5-e2a16555c8e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:08:24.585314 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: warnings.warn(msg) Jul 18 14:08:24.585314 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: Jul 18 14:08:24.587266 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce41c9-09e6-4976-97c5-e2a16555c8e4 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.600511 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 794/1604] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:08:24 2022] PUT /identity/v3/projects/1a0691238b54478d81de9ada168aea28/users/81cd7a7f6a9c4dbbaa3548cbca297762/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:24.601291 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce41c9-09e6-4976-97c5-e2a16555c8e4 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.650885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeb9162-8a8b-417c-85cb-62707a7dc890 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.664574 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce41c9-09e6-4976-97c5-e2a16555c8e4 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.670076 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c573e9a-7b0a-4bf0-9f1b-4c9f89a58841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.670963 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c573e9a-7b0a-4bf0-9f1b-4c9f89a58841 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.671246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c573e9a-7b0a-4bf0-9f1b-4c9f89a58841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.671526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c573e9a-7b0a-4bf0-9f1b-4c9f89a58841 admin admin] PATH_INFO: `/v3/projects/da8ad8d14f4a4e4bac2c0b60f7ccfcac/users/ac1a4dab03104482813134c664f34de6/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.686217 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c573e9a-7b0a-4bf0-9f1b-4c9f89a58841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da8ad8d14f4a4e4bac2c0b60f7ccfcac, user_id=ac1a4dab03104482813134c664f34de6, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.688923 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c573e9a-7b0a-4bf0-9f1b-4c9f89a58841 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.703177 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 795/1605] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:08:24 2022] PUT /identity/v3/projects/da8ad8d14f4a4e4bac2c0b60f7ccfcac/users/ac1a4dab03104482813134c664f34de6/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:24.704792 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c573e9a-7b0a-4bf0-9f1b-4c9f89a58841 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.709017 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c146a26-5b90-4397-bf89-da42872ab77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.710050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c146a26-5b90-4397-bf89-da42872ab77e admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.710671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c146a26-5b90-4397-bf89-da42872ab77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.710806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c146a26-5b90-4397-bf89-da42872ab77e admin admin] PATH_INFO: `/v3/projects/c9a6664b4d194755bc4b8e46ef187a7a/users/d3e29b6b60a848d59e23128b1ada1d9d/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.726488 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c146a26-5b90-4397-bf89-da42872ab77e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9a6664b4d194755bc4b8e46ef187a7a, user_id=d3e29b6b60a848d59e23128b1ada1d9d, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.729351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c146a26-5b90-4397-bf89-da42872ab77e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.732133 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:24.743995 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 811/1606] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:08:24 2022] PUT /identity/v3/projects/c9a6664b4d194755bc4b8e46ef187a7a/users/d3e29b6b60a848d59e23128b1ada1d9d/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:24.749257 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363a641-23c4-4310-b07b-f32382320f40 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.749559 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363a641-23c4-4310-b07b-f32382320f40 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.749920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363a641-23c4-4310-b07b-f32382320f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.766637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b363a641-23c4-4310-b07b-f32382320f40 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:24.767882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b363a641-23c4-4310-b07b-f32382320f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:24.771589 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c573e9a-7b0a-4bf0-9f1b-4c9f89a58841 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.817790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:24.827676 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 812/1607] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:24.828668 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363a641-23c4-4310-b07b-f32382320f40 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.834719 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6df5cc-11af-40cc-9770-b00806387792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.835348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6df5cc-11af-40cc-9770-b00806387792 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.835639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6df5cc-11af-40cc-9770-b00806387792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.835922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6df5cc-11af-40cc-9770-b00806387792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.836678 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6df5cc-11af-40cc-9770-b00806387792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.838357 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6df5cc-11af-40cc-9770-b00806387792 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.850189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 796/1608] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:24.851063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6df5cc-11af-40cc-9770-b00806387792 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.860619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6df5cc-11af-40cc-9770-b00806387792 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.869651 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1fba4-0c27-4c66-9547-58b532e4289c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.871410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1fba4-0c27-4c66-9547-58b532e4289c admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.871702 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1fba4-0c27-4c66-9547-58b532e4289c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.872336 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1fba4-0c27-4c66-9547-58b532e4289c admin admin] PATH_INFO: `/v3/projects/31755010e81b43908bbef9ae75eaaa64/users/0b6cb92018ea4cc5a25c5423b9bcd7a9/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.883682 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363a641-23c4-4310-b07b-f32382320f40 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.887599 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1fba4-0c27-4c66-9547-58b532e4289c admin admin] RBAC: Authorizing `identity:create_grant(project_id=31755010e81b43908bbef9ae75eaaa64, user_id=0b6cb92018ea4cc5a25c5423b9bcd7a9, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.888096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c432adc7-4ccd-4f5b-9713-69b3b6e66ae7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.888749 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432adc7-4ccd-4f5b-9713-69b3b6e66ae7 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.889038 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432adc7-4ccd-4f5b-9713-69b3b6e66ae7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.889322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432adc7-4ccd-4f5b-9713-69b3b6e66ae7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.890309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1fba4-0c27-4c66-9547-58b532e4289c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.898647 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c432adc7-4ccd-4f5b-9713-69b3b6e66ae7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.900122 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c432adc7-4ccd-4f5b-9713-69b3b6e66ae7 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.903536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 797/1609] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:08:24 2022] PUT /identity/v3/projects/31755010e81b43908bbef9ae75eaaa64/users/0b6cb92018ea4cc5a25c5423b9bcd7a9/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:24.907908 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1fba4-0c27-4c66-9547-58b532e4289c admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:24.910002 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 813/1610] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:24.911244 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8727cf-fb21-45c6-8c71-385c9ce4e3f2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.911570 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8727cf-fb21-45c6-8c71-385c9ce4e3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.911926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8727cf-fb21-45c6-8c71-385c9ce4e3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.951223 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2e8727cf-fb21-45c6-8c71-385c9ce4e3f2 None None] MFA Rules not processed for user `81cd7a7f6a9c4dbbaa3548cbca297762`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:24.953015 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2e8727cf-fb21-45c6-8c71-385c9ce4e3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:24.976269 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1fba4-0c27-4c66-9547-58b532e4289c admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:24.981538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d498001-9d6e-49c2-8db9-a88ed4f2dd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:24.982253 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d498001-9d6e-49c2-8db9-a88ed4f2dd5c admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.982577 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d498001-9d6e-49c2-8db9-a88ed4f2dd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.982898 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d498001-9d6e-49c2-8db9-a88ed4f2dd5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:24.983701 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d498001-9d6e-49c2-8db9-a88ed4f2dd5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:24.985510 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d498001-9d6e-49c2-8db9-a88ed4f2dd5c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:24.996695 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 798/1611] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:24.997999 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a855e-1d55-4a0f-8c13-f203dfaa30c4 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:24.998309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a855e-1d55-4a0f-8c13-f203dfaa30c4 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:24.998577 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a855e-1d55-4a0f-8c13-f203dfaa30c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:25.028998 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-bb9a855e-1d55-4a0f-8c13-f203dfaa30c4 None None] MFA Rules not processed for user `ac1a4dab03104482813134c664f34de6`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:25.030688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bb9a855e-1d55-4a0f-8c13-f203dfaa30c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:25.031991 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 814/1612] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:25.033478 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558bede4-2a2c-4e8a-bbcb-07f48a8264fd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:25.033773 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558bede4-2a2c-4e8a-bbcb-07f48a8264fd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:25.034049 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558bede4-2a2c-4e8a-bbcb-07f48a8264fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:25.061978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-558bede4-2a2c-4e8a-bbcb-07f48a8264fd None None] MFA Rules not processed for user `d3e29b6b60a848d59e23128b1ada1d9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:25.063686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-558bede4-2a2c-4e8a-bbcb-07f48a8264fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:25.080191 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:25.107887 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 799/1613] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:25.108843 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a855e-1d55-4a0f-8c13-f203dfaa30c4 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:25.146023 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 815/1614] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:25.147174 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558bede4-2a2c-4e8a-bbcb-07f48a8264fd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:25.181029 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:25.186266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558bede4-2a2c-4e8a-bbcb-07f48a8264fd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:25.197462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a855e-1d55-4a0f-8c13-f203dfaa30c4 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:25.200847 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb78cc-2fb2-446e-92b0-be8a783b7231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:25.201760 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb78cc-2fb2-446e-92b0-be8a783b7231 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:25.202054 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb78cc-2fb2-446e-92b0-be8a783b7231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:25.202368 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb78cc-2fb2-446e-92b0-be8a783b7231 admin admin] PATH_INFO: `/v3/projects/799dde59367144af9f1f8cce1cb7d435/users/226d34540e464a93aa984ff59b497450/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:25.202918 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ab153-f7dc-4487-8d3e-cef9c3492295 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:25.206897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ab153-f7dc-4487-8d3e-cef9c3492295 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:25.206897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ab153-f7dc-4487-8d3e-cef9c3492295 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:25.206897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ab153-f7dc-4487-8d3e-cef9c3492295 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:25.227574 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ab153-f7dc-4487-8d3e-cef9c3492295 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:25.231710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ab153-f7dc-4487-8d3e-cef9c3492295 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:25.232952 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb78cc-2fb2-446e-92b0-be8a783b7231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=799dde59367144af9f1f8cce1cb7d435, user_id=226d34540e464a93aa984ff59b497450, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:25.236235 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb78cc-2fb2-446e-92b0-be8a783b7231 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:25.247437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 800/1615] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:25.255727 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39391c6-0ed0-4857-87d6-03b5ec23ba0e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:25.256164 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39391c6-0ed0-4857-87d6-03b5ec23ba0e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:25.256401 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 816/1616] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:08:25 2022] PUT /identity/v3/projects/799dde59367144af9f1f8cce1cb7d435/users/226d34540e464a93aa984ff59b497450/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:25.257583 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb78cc-2fb2-446e-92b0-be8a783b7231 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:25.261198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39391c6-0ed0-4857-87d6-03b5ec23ba0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:25.287629 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b39391c6-0ed0-4857-87d6-03b5ec23ba0e None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:25.290301 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b39391c6-0ed0-4857-87d6-03b5ec23ba0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:25.339411 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb78cc-2fb2-446e-92b0-be8a783b7231 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:25.344341 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084ef6e-1a3d-45c4-a362-ef7c5df9e734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:25.345527 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084ef6e-1a3d-45c4-a362-ef7c5df9e734 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:25.345677 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084ef6e-1a3d-45c4-a362-ef7c5df9e734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:25.346019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084ef6e-1a3d-45c4-a362-ef7c5df9e734 admin admin] PATH_INFO: `/v3/projects/c70edb2715d64ceaab313111a78ed840/users/d9fbda50766d4ca98dafeedbfa2c36d2/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:25.349282 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f084ef6e-1a3d-45c4-a362-ef7c5df9e734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c70edb2715d64ceaab313111a78ed840, user_id=d9fbda50766d4ca98dafeedbfa2c36d2, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:25.352197 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f084ef6e-1a3d-45c4-a362-ef7c5df9e734 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:25.365491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 817/1617] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:08:25 2022] PUT /identity/v3/projects/c70edb2715d64ceaab313111a78ed840/users/d9fbda50766d4ca98dafeedbfa2c36d2/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:25.366318 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084ef6e-1a3d-45c4-a362-ef7c5df9e734 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:25.417236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 801/1618] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:25.423659 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889ca7d-9056-4c97-b71a-50f1e60d9a37 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:25.423972 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889ca7d-9056-4c97-b71a-50f1e60d9a37 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:25.424263 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889ca7d-9056-4c97-b71a-50f1e60d9a37 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:25.428077 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 802/1619] 10.222.0.94 () {54 vars in 962 bytes} [Mon Jul 18 14:08:25 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:08:25.433501 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317623b6-7ab2-4b09-b03b-2c0314a5ffba None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:25.434130 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317623b6-7ab2-4b09-b03b-2c0314a5ffba None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:25.435035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317623b6-7ab2-4b09-b03b-2c0314a5ffba None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:25.437456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 803/1620] 10.222.0.94 () {54 vars in 962 bytes} [Mon Jul 18 14:08:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:08:25.441325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a90e1e-daba-4cf7-8664-0cc5c6596dcc None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:25.442040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a90e1e-daba-4cf7-8664-0cc5c6596dcc None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:25.442851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a90e1e-daba-4cf7-8664-0cc5c6596dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:25.473139 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-a8a90e1e-daba-4cf7-8664-0cc5c6596dcc None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:25.474378 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a8a90e1e-daba-4cf7-8664-0cc5c6596dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:25.479352 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084ef6e-1a3d-45c4-a362-ef7c5df9e734 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:25.486755 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a04432-1026-4115-a457-95feccdf52c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:25.487172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a04432-1026-4115-a457-95feccdf52c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:25.487381 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a04432-1026-4115-a457-95feccdf52c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:25.487683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a04432-1026-4115-a457-95feccdf52c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:25.488386 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a04432-1026-4115-a457-95feccdf52c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:25.490312 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a04432-1026-4115-a457-95feccdf52c4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:25.500539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 804/1621] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:25.502238 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5662ba9-f548-4881-b439-d9bbf878515a None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:25.503287 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5662ba9-f548-4881-b439-d9bbf878515a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:25.503287 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5662ba9-f548-4881-b439-d9bbf878515a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:25.531982 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 818/1622] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:08:25 2022] POST /identity/v3/projects => generated 365 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:25.535388 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a04432-1026-4115-a457-95feccdf52c4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:25.538972 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b5662ba9-f548-4881-b439-d9bbf878515a None None] MFA Rules not processed for user `d9fbda50766d4ca98dafeedbfa2c36d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:25.540774 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b5662ba9-f548-4881-b439-d9bbf878515a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:25.576253 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:25.634842 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a04432-1026-4115-a457-95feccdf52c4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:25.639828 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806823d-2700-49f2-aa64-1f043e5831c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:25.640555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806823d-2700-49f2-aa64-1f043e5831c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:25.640877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806823d-2700-49f2-aa64-1f043e5831c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:25.641166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806823d-2700-49f2-aa64-1f043e5831c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:25.654291 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 805/1623] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:25.655800 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0335dd35-b016-401a-97cd-718810de7d73 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:25.656126 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0335dd35-b016-401a-97cd-718810de7d73 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:25.656419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0335dd35-b016-401a-97cd-718810de7d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:25.701093 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-0335dd35-b016-401a-97cd-718810de7d73 None None] MFA Rules not processed for user `ac717fcd9fc1499db12cf852d45572a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:25.702320 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0335dd35-b016-401a-97cd-718810de7d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:25.713978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806823d-2700-49f2-aa64-1f043e5831c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:25.715254 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806823d-2700-49f2-aa64-1f043e5831c9 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:25.725690 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 819/1624] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:25.727062 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda0727-f938-49a6-8d83-6f2b3fa7cb26 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:25.727359 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda0727-f938-49a6-8d83-6f2b3fa7cb26 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:25.727643 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda0727-f938-49a6-8d83-6f2b3fa7cb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:25.746732 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9bda0727-f938-49a6-8d83-6f2b3fa7cb26 None None] MFA Rules not processed for user `ac717fcd9fc1499db12cf852d45572a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:25.748096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9bda0727-f938-49a6-8d83-6f2b3fa7cb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:25.762165 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:25.785641 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 806/1625] 10.222.0.94 () {58 vars in 1059 bytes} [Mon Jul 18 14:08:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:25.786518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0335dd35-b016-401a-97cd-718810de7d73 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:25.802259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0335dd35-b016-401a-97cd-718810de7d73 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:25.808234 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a010d-7f2d-430c-8d4a-a8615ebdc7a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:25.809169 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a010d-7f2d-430c-8d4a-a8615ebdc7a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:25.809793 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a010d-7f2d-430c-8d4a-a8615ebdc7a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:25.810087 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a010d-7f2d-430c-8d4a-a8615ebdc7a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:25.833137 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a010d-7f2d-430c-8d4a-a8615ebdc7a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:25.834220 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 820/1626] 10.222.0.94 () {58 vars in 1059 bytes} [Mon Jul 18 14:08:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:25.834611 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a010d-7f2d-430c-8d4a-a8615ebdc7a3 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:25.835837 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda0727-f938-49a6-8d83-6f2b3fa7cb26 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:25.848725 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda0727-f938-49a6-8d83-6f2b3fa7cb26 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:25.851257 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 807/1627] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:25.853822 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a010d-7f2d-430c-8d4a-a8615ebdc7a3 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:25.874191 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4928a-26d2-43a4-9c36-c8932b73ee5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:25.874918 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4928a-26d2-43a4-9c36-c8932b73ee5a admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:25.875236 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4928a-26d2-43a4-9c36-c8932b73ee5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:25.877661 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4928a-26d2-43a4-9c36-c8932b73ee5a admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:25.878463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b4928a-26d2-43a4-9c36-c8932b73ee5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:25.881220 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b4928a-26d2-43a4-9c36-c8932b73ee5a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:25.930803 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 821/1628] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:08:25 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:25.933547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4928a-26d2-43a4-9c36-c8932b73ee5a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:25.958740 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4928a-26d2-43a4-9c36-c8932b73ee5a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:25.971746 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee05479-5a73-423d-af24-5b28c72a921f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:25.976203 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee05479-5a73-423d-af24-5b28c72a921f service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:25.976916 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee05479-5a73-423d-af24-5b28c72a921f service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:25.979052 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee05479-5a73-423d-af24-5b28c72a921f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:26.003787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a010d-7f2d-430c-8d4a-a8615ebdc7a3 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:26.014900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc23e81-3d7b-4640-b607-f1ec3a2bde42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:26.015666 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc23e81-3d7b-4640-b607-f1ec3a2bde42 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:26.015981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc23e81-3d7b-4640-b607-f1ec3a2bde42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:26.017452 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc23e81-3d7b-4640-b607-f1ec3a2bde42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:26.042267 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc23e81-3d7b-4640-b607-f1ec3a2bde42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:26.051069 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc23e81-3d7b-4640-b607-f1ec3a2bde42 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:26.059742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:26.073740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 808/1629] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:25 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:26.074629 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc23e81-3d7b-4640-b607-f1ec3a2bde42 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:26.092135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc23e81-3d7b-4640-b607-f1ec3a2bde42 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:26.103621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bed7db-23b1-4de4-a39f-a50848e32f69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:26.104325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bed7db-23b1-4de4-a39f-a50848e32f69 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:26.104700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bed7db-23b1-4de4-a39f-a50848e32f69 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:26.104988 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bed7db-23b1-4de4-a39f-a50848e32f69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:26.134250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee05479-5a73-423d-af24-5b28c72a921f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:26.139519 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee05479-5a73-423d-af24-5b28c72a921f service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:26.167034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 822/1630] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:26.168234 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee05479-5a73-423d-af24-5b28c72a921f service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:26.185222 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee05479-5a73-423d-af24-5b28c72a921f service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:26.202103 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d3d48-8a79-4a1a-b7be-249ce0faa348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:26.202736 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d3d48-8a79-4a1a-b7be-249ce0faa348 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:26.203126 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d3d48-8a79-4a1a-b7be-249ce0faa348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:26.203531 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d3d48-8a79-4a1a-b7be-249ce0faa348 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:26.204552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d3d48-8a79-4a1a-b7be-249ce0faa348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:26.207092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d3d48-8a79-4a1a-b7be-249ce0faa348 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:26.237834 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 823/1631] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:26.242069 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d3d48-8a79-4a1a-b7be-249ce0faa348 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:26.253685 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bed7db-23b1-4de4-a39f-a50848e32f69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:26.255432 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bed7db-23b1-4de4-a39f-a50848e32f69 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:26.260211 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d3d48-8a79-4a1a-b7be-249ce0faa348 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:26.268706 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b02a2-1039-4e69-baa6-6185f73bc093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:26.269848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b02a2-1039-4e69-baa6-6185f73bc093 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:26.270276 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b02a2-1039-4e69-baa6-6185f73bc093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:26.270691 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b02a2-1039-4e69-baa6-6185f73bc093 admin admin] PATH_INFO: `/v3/projects/ca6f8b8ef21b4621ab2e2eab6096396e/users/d0524f4f86bc4ed388a573427e3705dd/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:26.278914 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 809/1632] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:26.280839 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bed7db-23b1-4de4-a39f-a50848e32f69 service nova] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:26.296353 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b02a2-1039-4e69-baa6-6185f73bc093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca6f8b8ef21b4621ab2e2eab6096396e, user_id=d0524f4f86bc4ed388a573427e3705dd, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:26.301339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b02a2-1039-4e69-baa6-6185f73bc093 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:26.302833 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bed7db-23b1-4de4-a39f-a50848e32f69 service nova] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:26.315897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba0d93-b20d-4f18-945f-cd5b0b2310ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:26.316704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba0d93-b20d-4f18-945f-cd5b0b2310ca service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:26.317073 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba0d93-b20d-4f18-945f-cd5b0b2310ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:26.317699 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba0d93-b20d-4f18-945f-cd5b0b2310ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:26.348569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 824/1633] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:08:26 2022] PUT /identity/v3/projects/ca6f8b8ef21b4621ab2e2eab6096396e/users/d0524f4f86bc4ed388a573427e3705dd/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:08:26.350579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b02a2-1039-4e69-baa6-6185f73bc093 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:26.446407 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba0d93-b20d-4f18-945f-cd5b0b2310ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:26.449518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba0d93-b20d-4f18-945f-cd5b0b2310ca service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:26.472527 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 810/1634] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:26.521647 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b02a2-1039-4e69-baa6-6185f73bc093 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:26.530252 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92430258-f4a7-4498-9ed3-9884c50c4e3e None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:26.530758 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92430258-f4a7-4498-9ed3-9884c50c4e3e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:26.531197 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92430258-f4a7-4498-9ed3-9884c50c4e3e None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:26.533021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 811/1635] 10.222.0.94 () {54 vars in 912 bytes} [Mon Jul 18 14:08:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:08:26.539538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711f4bf-9d63-4de5-ad98-f866a7884679 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:26.539538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711f4bf-9d63-4de5-ad98-f866a7884679 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:26.540299 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711f4bf-9d63-4de5-ad98-f866a7884679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:26.544231 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458fbc7-b190-4c9e-9581-675e6fc2e951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:26.544981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458fbc7-b190-4c9e-9581-675e6fc2e951 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:26.545302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458fbc7-b190-4c9e-9581-675e6fc2e951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:26.550024 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458fbc7-b190-4c9e-9581-675e6fc2e951 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:26.553208 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458fbc7-b190-4c9e-9581-675e6fc2e951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:26.555028 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458fbc7-b190-4c9e-9581-675e6fc2e951 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:26.567819 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 825/1636] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:08:26.571594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458fbc7-b190-4c9e-9581-675e6fc2e951 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:26.592414 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-0711f4bf-9d63-4de5-ad98-f866a7884679 None None] MFA Rules not processed for user `ac717fcd9fc1499db12cf852d45572a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:26.594370 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458fbc7-b190-4c9e-9581-675e6fc2e951 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:26.599206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0711f4bf-9d63-4de5-ad98-f866a7884679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:26.605991 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecad4da5-2bec-4340-8ec7-37a8a13bbe40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:26.607627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad4da5-2bec-4340-8ec7-37a8a13bbe40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:26.609815 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad4da5-2bec-4340-8ec7-37a8a13bbe40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:26.610854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad4da5-2bec-4340-8ec7-37a8a13bbe40 admin admin] PATH_INFO: `/v3/projects/ca6f8b8ef21b4621ab2e2eab6096396e/users/d0524f4f86bc4ed388a573427e3705dd/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:26.617354 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad4da5-2bec-4340-8ec7-37a8a13bbe40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca6f8b8ef21b4621ab2e2eab6096396e, user_id=d0524f4f86bc4ed388a573427e3705dd, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:26.623124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:26.625322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecad4da5-2bec-4340-8ec7-37a8a13bbe40 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:26.670338 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 826/1637] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:08:26 2022] PUT /identity/v3/projects/ca6f8b8ef21b4621ab2e2eab6096396e/users/d0524f4f86bc4ed388a573427e3705dd/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:26.672825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b53ba4-97d4-4658-9e5d-65fca10b532f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:26.673150 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b53ba4-97d4-4658-9e5d-65fca10b532f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:26.674335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b53ba4-97d4-4658-9e5d-65fca10b532f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:26.678648 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 827/1638] 10.222.0.94 () {54 vars in 912 bytes} [Mon Jul 18 14:08:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:08:26.682212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd484c8-1bf0-49f1-a294-186491aac969 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:26.683426 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd484c8-1bf0-49f1-a294-186491aac969 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:26.683750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd484c8-1bf0-49f1-a294-186491aac969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:26.757025 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5fd484c8-1bf0-49f1-a294-186491aac969 None None] MFA Rules not processed for user `d0524f4f86bc4ed388a573427e3705dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:26.765366 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5fd484c8-1bf0-49f1-a294-186491aac969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:26.770226 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 812/1639] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Jul 18 14:08:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:26.771272 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63975ef-3638-4877-b6df-113f985878da None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:26.771610 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63975ef-3638-4877-b6df-113f985878da None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:26.771849 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63975ef-3638-4877-b6df-113f985878da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:26.806243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c63975ef-3638-4877-b6df-113f985878da None None] MFA Rules not processed for user `ac717fcd9fc1499db12cf852d45572a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:26.808048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c63975ef-3638-4877-b6df-113f985878da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:26.957472 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 828/1640] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:26 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:26.958393 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd484c8-1bf0-49f1-a294-186491aac969 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:26.965577 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 813/1641] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Jul 18 14:08:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:26.974417 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:26.999128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63975ef-3638-4877-b6df-113f985878da None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:27.047711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd484c8-1bf0-49f1-a294-186491aac969 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:27.052949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2931948-ce66-4194-9785-2bb5709acfe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:27.054084 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2931948-ce66-4194-9785-2bb5709acfe5 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:27.054379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2931948-ce66-4194-9785-2bb5709acfe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:27.054663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2931948-ce66-4194-9785-2bb5709acfe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:27.080871 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2931948-ce66-4194-9785-2bb5709acfe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:27.082444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2931948-ce66-4194-9785-2bb5709acfe5 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:27.110740 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 829/1642] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:27.121562 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63975ef-3638-4877-b6df-113f985878da None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:27.127103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324baa8-cee3-4ca1-9361-7bc4b569db6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:27.128625 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324baa8-cee3-4ca1-9361-7bc4b569db6c service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:27.128889 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324baa8-cee3-4ca1-9361-7bc4b569db6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:27.129194 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324baa8-cee3-4ca1-9361-7bc4b569db6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:27.164753 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a324baa8-cee3-4ca1-9361-7bc4b569db6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:27.169101 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a324baa8-cee3-4ca1-9361-7bc4b569db6c service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:27.189563 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 814/1643] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:26 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:08:27.286099 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2931948-ce66-4194-9785-2bb5709acfe5 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:27.297212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2931948-ce66-4194-9785-2bb5709acfe5 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:27.305464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f6a46-d837-4138-bc94-7aacb1bb00f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:27.307575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f6a46-d837-4138-bc94-7aacb1bb00f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:27.307864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f6a46-d837-4138-bc94-7aacb1bb00f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:27.309932 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f6a46-d837-4138-bc94-7aacb1bb00f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:27.336069 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f6a46-d837-4138-bc94-7aacb1bb00f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:27.340150 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f6a46-d837-4138-bc94-7aacb1bb00f0 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:27.381117 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 830/1644] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:29.573752 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324baa8-cee3-4ca1-9361-7bc4b569db6c service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:29.645580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:29.675673 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324baa8-cee3-4ca1-9361-7bc4b569db6c service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:29.682204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b295a00-00e7-47e2-90d8-b3e56d456bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:29.683505 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b295a00-00e7-47e2-90d8-b3e56d456bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:29.683900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b295a00-00e7-47e2-90d8-b3e56d456bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:29.684490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b295a00-00e7-47e2-90d8-b3e56d456bf1 admin admin] PATH_INFO: `/v3/users/81cd7a7f6a9c4dbbaa3548cbca297762` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:29.686550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b295a00-00e7-47e2-90d8-b3e56d456bf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81cd7a7f6a9c4dbbaa3548cbca297762)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:29.687976 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: WARNING py.warnings [None req-8b295a00-00e7-47e2-90d8-b3e56d456bf1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:08:29.687976 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: warnings.warn(msg) Jul 18 14:08:29.687976 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: Jul 18 14:08:29.689727 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b295a00-00e7-47e2-90d8-b3e56d456bf1 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:29.802941 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8b295a00-00e7-47e2-90d8-b3e56d456bf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81cd7a7f6a9c4dbbaa3548cbca297762'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:29.815080 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8b295a00-00e7-47e2-90d8-b3e56d456bf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81cd7a7f6a9c4dbbaa3548cbca297762'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:29.845304 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8b295a00-00e7-47e2-90d8-b3e56d456bf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81cd7a7f6a9c4dbbaa3548cbca297762'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:29.846167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8b295a00-00e7-47e2-90d8-b3e56d456bf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81cd7a7f6a9c4dbbaa3548cbca297762'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:29.877308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8b295a00-00e7-47e2-90d8-b3e56d456bf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81cd7a7f6a9c4dbbaa3548cbca297762'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:29.890129 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 815/1645] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:08:29 2022] DELETE /identity/v3/users/81cd7a7f6a9c4dbbaa3548cbca297762 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:29.892328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc4a72-962c-4610-8990-51f06cc6c3cc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:29.892888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc4a72-962c-4610-8990-51f06cc6c3cc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:29.892975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc4a72-962c-4610-8990-51f06cc6c3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:29.922724 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-24fc4a72-962c-4610-8990-51f06cc6c3cc None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:29.925141 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-24fc4a72-962c-4610-8990-51f06cc6c3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:30.157767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 831/1646] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:30.219907 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b295a00-00e7-47e2-90d8-b3e56d456bf1 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:30.224283 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8b295a00-00e7-47e2-90d8-b3e56d456bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:30.340637 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b295a00-00e7-47e2-90d8-b3e56d456bf1 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:30.347247 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38006b-f7fd-473d-b9b9-642962b2c8a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:30.348868 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38006b-f7fd-473d-b9b9-642962b2c8a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:30.349622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38006b-f7fd-473d-b9b9-642962b2c8a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:30.350144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38006b-f7fd-473d-b9b9-642962b2c8a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:30.372799 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b38006b-f7fd-473d-b9b9-642962b2c8a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:30.374742 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b38006b-f7fd-473d-b9b9-642962b2c8a2 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:30.386959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 816/1647] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:32.064769 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc4a72-962c-4610-8990-51f06cc6c3cc None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:32.067710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-24fc4a72-962c-4610-8990-51f06cc6c3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:32.087303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc4a72-962c-4610-8990-51f06cc6c3cc None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:32.119316 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7bb0a-90ec-4d71-9ecc-f8f8dd8d6455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:32.119316 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7bb0a-90ec-4d71-9ecc-f8f8dd8d6455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:32.120623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7bb0a-90ec-4d71-9ecc-f8f8dd8d6455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:32.120623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf7bb0a-90ec-4d71-9ecc-f8f8dd8d6455 admin admin] PATH_INFO: `/v3/projects/1a0691238b54478d81de9ada168aea28` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:32.128065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7bb0a-90ec-4d71-9ecc-f8f8dd8d6455 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a0691238b54478d81de9ada168aea28)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:32.129145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-fbf7bb0a-90ec-4d71-9ecc-f8f8dd8d6455 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:08:32.129145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Jul 18 14:08:32.129145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: Jul 18 14:08:32.130156 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf7bb0a-90ec-4d71-9ecc-f8f8dd8d6455 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:32.169988 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38006b-f7fd-473d-b9b9-642962b2c8a2 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:32.173668 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9b38006b-f7fd-473d-b9b9-642962b2c8a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:32.196199 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38006b-f7fd-473d-b9b9-642962b2c8a2 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:32.205582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4468c-18d8-4dd3-98ee-74b9807a83f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:32.206603 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4468c-18d8-4dd3-98ee-74b9807a83f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:32.207008 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4468c-18d8-4dd3-98ee-74b9807a83f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:32.207378 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4468c-18d8-4dd3-98ee-74b9807a83f2 admin admin] PATH_INFO: `/v3/users/ac1a4dab03104482813134c664f34de6` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:32.211990 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4468c-18d8-4dd3-98ee-74b9807a83f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac1a4dab03104482813134c664f34de6)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:32.214692 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4468c-18d8-4dd3-98ee-74b9807a83f2 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:32.303176 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fbf7bb0a-90ec-4d71-9ecc-f8f8dd8d6455 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a0691238b54478d81de9ada168aea28', 'request_id': 'req-fbf7bb0a-90ec-4d71-9ecc-f8f8dd8d6455'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:32.344668 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fbf7bb0a-90ec-4d71-9ecc-f8f8dd8d6455 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a0691238b54478d81de9ada168aea28', 'request_id': 'req-fbf7bb0a-90ec-4d71-9ecc-f8f8dd8d6455'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:32.374621 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fbf7bb0a-90ec-4d71-9ecc-f8f8dd8d6455 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a0691238b54478d81de9ada168aea28', 'request_id': 'req-fbf7bb0a-90ec-4d71-9ecc-f8f8dd8d6455'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:32.378475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c3e4468c-18d8-4dd3-98ee-74b9807a83f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac1a4dab03104482813134c664f34de6'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:32.393852 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fbf7bb0a-90ec-4d71-9ecc-f8f8dd8d6455 admin admin] The token cache is being invalidate because project 1a0691238b54478d81de9ada168aea28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:08:32.394017 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fbf7bb0a-90ec-4d71-9ecc-f8f8dd8d6455 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:32.395656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fbf7bb0a-90ec-4d71-9ecc-f8f8dd8d6455 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:32.398063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c3e4468c-18d8-4dd3-98ee-74b9807a83f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac1a4dab03104482813134c664f34de6'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:32.399756 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:32.416126 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 832/1648] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:08:32 2022] DELETE /identity/v3/projects/1a0691238b54478d81de9ada168aea28 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:32.418138 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c3e4468c-18d8-4dd3-98ee-74b9807a83f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac1a4dab03104482813134c664f34de6'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:32.418250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7bb0a-90ec-4d71-9ecc-f8f8dd8d6455 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:32.419621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c3e4468c-18d8-4dd3-98ee-74b9807a83f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac1a4dab03104482813134c664f34de6'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:32.425493 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fbf7bb0a-90ec-4d71-9ecc-f8f8dd8d6455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:32.454462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c3e4468c-18d8-4dd3-98ee-74b9807a83f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac1a4dab03104482813134c664f34de6'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:32.480239 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 817/1649] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:08:32 2022] DELETE /identity/v3/users/ac1a4dab03104482813134c664f34de6 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:08:32.481840 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cd5fd-eaa4-415a-a765-3bd4f0d29d34 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:32.482145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cd5fd-eaa4-415a-a765-3bd4f0d29d34 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:32.482434 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cd5fd-eaa4-415a-a765-3bd4f0d29d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:32.488179 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:32.518049 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-805cd5fd-eaa4-415a-a765-3bd4f0d29d34 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:32.518838 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-805cd5fd-eaa4-415a-a765-3bd4f0d29d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:32.591734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf7bb0a-90ec-4d71-9ecc-f8f8dd8d6455 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:32.696104 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 818/1650] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:32.731624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7ef77-4f93-4dcd-8166-22215dec9bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:32.732793 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7ef77-4f93-4dcd-8166-22215dec9bde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:32.733154 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7ef77-4f93-4dcd-8166-22215dec9bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:32.733600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7ef77-4f93-4dcd-8166-22215dec9bde admin admin] PATH_INFO: `/v3/users/d0524f4f86bc4ed388a573427e3705dd` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:32.734944 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe7ef77-4f93-4dcd-8166-22215dec9bde admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0524f4f86bc4ed388a573427e3705dd)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:32.737185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe7ef77-4f93-4dcd-8166-22215dec9bde admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:32.820317 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1fe7ef77-4f93-4dcd-8166-22215dec9bde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0524f4f86bc4ed388a573427e3705dd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:32.839698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1fe7ef77-4f93-4dcd-8166-22215dec9bde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0524f4f86bc4ed388a573427e3705dd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:32.854454 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1fe7ef77-4f93-4dcd-8166-22215dec9bde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0524f4f86bc4ed388a573427e3705dd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:32.869237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1fe7ef77-4f93-4dcd-8166-22215dec9bde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0524f4f86bc4ed388a573427e3705dd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:32.918804 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805cd5fd-eaa4-415a-a765-3bd4f0d29d34 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:32.920670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-805cd5fd-eaa4-415a-a765-3bd4f0d29d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:32.924876 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1fe7ef77-4f93-4dcd-8166-22215dec9bde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0524f4f86bc4ed388a573427e3705dd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:32.941598 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 833/1651] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:08:32 2022] DELETE /identity/v3/users/d0524f4f86bc4ed388a573427e3705dd => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:33.035104 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805cd5fd-eaa4-415a-a765-3bd4f0d29d34 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:33.093873 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516797da-dccb-45e4-aae6-a612ccc257f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:33.094766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516797da-dccb-45e4-aae6-a612ccc257f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:33.095138 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516797da-dccb-45e4-aae6-a612ccc257f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:33.095494 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516797da-dccb-45e4-aae6-a612ccc257f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:33.098243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-516797da-dccb-45e4-aae6-a612ccc257f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:33.152218 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:33.178614 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516797da-dccb-45e4-aae6-a612ccc257f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:33.183676 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516797da-dccb-45e4-aae6-a612ccc257f7 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:33.199509 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 819/1652] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:34.389599 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7ef77-4f93-4dcd-8166-22215dec9bde admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:34.391799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1fe7ef77-4f93-4dcd-8166-22215dec9bde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:34.410292 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7ef77-4f93-4dcd-8166-22215dec9bde admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:34.426155 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5dc44c-c7c2-4808-92ef-9c7cf27f8c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:34.427078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dc44c-c7c2-4808-92ef-9c7cf27f8c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:34.427372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dc44c-c7c2-4808-92ef-9c7cf27f8c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:34.427658 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5dc44c-c7c2-4808-92ef-9c7cf27f8c4b admin admin] PATH_INFO: `/v3/projects/ca6f8b8ef21b4621ab2e2eab6096396e` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:34.430576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5dc44c-c7c2-4808-92ef-9c7cf27f8c4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca6f8b8ef21b4621ab2e2eab6096396e)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:34.438604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5dc44c-c7c2-4808-92ef-9c7cf27f8c4b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:34.518844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:34.531925 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cb5dc44c-c7c2-4808-92ef-9c7cf27f8c4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca6f8b8ef21b4621ab2e2eab6096396e', 'request_id': 'req-cb5dc44c-c7c2-4808-92ef-9c7cf27f8c4b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:34.543186 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cb5dc44c-c7c2-4808-92ef-9c7cf27f8c4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca6f8b8ef21b4621ab2e2eab6096396e', 'request_id': 'req-cb5dc44c-c7c2-4808-92ef-9c7cf27f8c4b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:34.556175 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cb5dc44c-c7c2-4808-92ef-9c7cf27f8c4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca6f8b8ef21b4621ab2e2eab6096396e', 'request_id': 'req-cb5dc44c-c7c2-4808-92ef-9c7cf27f8c4b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:34.580240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cb5dc44c-c7c2-4808-92ef-9c7cf27f8c4b admin admin] The token cache is being invalidate because project ca6f8b8ef21b4621ab2e2eab6096396e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:08:34.580575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cb5dc44c-c7c2-4808-92ef-9c7cf27f8c4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:34.581207 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cb5dc44c-c7c2-4808-92ef-9c7cf27f8c4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:34.583338 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 834/1653] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:08:34 2022] DELETE /identity/v3/projects/ca6f8b8ef21b4621ab2e2eab6096396e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:34.589356 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10266cdb-b72b-4021-a3f5-f3cdc5db14a2 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:34.589847 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10266cdb-b72b-4021-a3f5-f3cdc5db14a2 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:34.589936 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10266cdb-b72b-4021-a3f5-f3cdc5db14a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:34.634170 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-10266cdb-b72b-4021-a3f5-f3cdc5db14a2 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:34.636058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-10266cdb-b72b-4021-a3f5-f3cdc5db14a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:34.809072 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 820/1654] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:34.810419 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1b9f6-05f2-4574-b2bf-a321fd268462 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:34.811439 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1b9f6-05f2-4574-b2bf-a321fd268462 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:34.811874 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1b9f6-05f2-4574-b2bf-a321fd268462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:34.831007 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10266cdb-b72b-4021-a3f5-f3cdc5db14a2 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:34.832491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-10266cdb-b72b-4021-a3f5-f3cdc5db14a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:34.840062 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e7f1b9f6-05f2-4574-b2bf-a321fd268462 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:34.848961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e7f1b9f6-05f2-4574-b2bf-a321fd268462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:34.853131 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10266cdb-b72b-4021-a3f5-f3cdc5db14a2 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:34.864943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9dbf6-f8ba-4f5e-9ccd-ea3737d878ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:34.865836 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9dbf6-f8ba-4f5e-9ccd-ea3737d878ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:34.866129 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9dbf6-f8ba-4f5e-9ccd-ea3737d878ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:34.866424 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9dbf6-f8ba-4f5e-9ccd-ea3737d878ab admin admin] PATH_INFO: `/v3/projects/da8ad8d14f4a4e4bac2c0b60f7ccfcac` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:34.869079 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b9dbf6-f8ba-4f5e-9ccd-ea3737d878ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=da8ad8d14f4a4e4bac2c0b60f7ccfcac)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:34.871492 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 835/1655] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:08:34.874723 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b9dbf6-f8ba-4f5e-9ccd-ea3737d878ab admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:34.875801 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1b9f6-05f2-4574-b2bf-a321fd268462 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:34.895232 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1b9f6-05f2-4574-b2bf-a321fd268462 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:34.913626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bbc704-4391-4fb4-8e20-70b7715dddab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:34.914477 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bbc704-4391-4fb4-8e20-70b7715dddab admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:34.915489 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bbc704-4391-4fb4-8e20-70b7715dddab admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:34.915875 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bbc704-4391-4fb4-8e20-70b7715dddab admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:34.917308 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bbc704-4391-4fb4-8e20-70b7715dddab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:34.917724 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bbc704-4391-4fb4-8e20-70b7715dddab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:34.919483 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bbc704-4391-4fb4-8e20-70b7715dddab admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:34.939418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 836/1656] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:34.945573 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bbc704-4391-4fb4-8e20-70b7715dddab admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:34.990415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-00b9dbf6-f8ba-4f5e-9ccd-ea3737d878ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da8ad8d14f4a4e4bac2c0b60f7ccfcac', 'request_id': 'req-00b9dbf6-f8ba-4f5e-9ccd-ea3737d878ab'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:34.992428 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bbc704-4391-4fb4-8e20-70b7715dddab admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:35.001097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-00b9dbf6-f8ba-4f5e-9ccd-ea3737d878ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da8ad8d14f4a4e4bac2c0b60f7ccfcac', 'request_id': 'req-00b9dbf6-f8ba-4f5e-9ccd-ea3737d878ab'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:35.017901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-00b9dbf6-f8ba-4f5e-9ccd-ea3737d878ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da8ad8d14f4a4e4bac2c0b60f7ccfcac', 'request_id': 'req-00b9dbf6-f8ba-4f5e-9ccd-ea3737d878ab'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:35.041861 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-00b9dbf6-f8ba-4f5e-9ccd-ea3737d878ab admin admin] The token cache is being invalidate because project da8ad8d14f4a4e4bac2c0b60f7ccfcac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:08:35.042157 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-00b9dbf6-f8ba-4f5e-9ccd-ea3737d878ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:35.044684 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-00b9dbf6-f8ba-4f5e-9ccd-ea3737d878ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:35.053094 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 821/1657] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:08:34 2022] DELETE /identity/v3/projects/da8ad8d14f4a4e4bac2c0b60f7ccfcac => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:08:35.056001 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8043a-ee39-4e65-8a73-5025312039e0 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:35.056319 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8043a-ee39-4e65-8a73-5025312039e0 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:35.056619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8043a-ee39-4e65-8a73-5025312039e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:35.105561 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-4ac8043a-ee39-4e65-8a73-5025312039e0 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:35.106817 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4ac8043a-ee39-4e65-8a73-5025312039e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:35.142037 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c3715-cf58-474b-ac89-6484b0bc54c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:35.142735 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c3715-cf58-474b-ac89-6484b0bc54c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:35.143030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c3715-cf58-474b-ac89-6484b0bc54c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:35.143315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c3715-cf58-474b-ac89-6484b0bc54c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:35.144052 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c3715-cf58-474b-ac89-6484b0bc54c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:35.146022 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c3715-cf58-474b-ac89-6484b0bc54c8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:35.236667 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 837/1658] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:08:34 2022] POST /identity/v3/projects => generated 381 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:08:35.243060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c3715-cf58-474b-ac89-6484b0bc54c8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:35.246469 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7b5c3715-cf58-474b-ac89-6484b0bc54c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:35.291959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 822/1659] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:35.294112 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a209a6e-6d72-4e1e-ada2-f88826b4e8ee None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:35.294112 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a209a6e-6d72-4e1e-ada2-f88826b4e8ee None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:35.294300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a209a6e-6d72-4e1e-ada2-f88826b4e8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:35.318046 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:35.353155 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-7a209a6e-6d72-4e1e-ada2-f88826b4e8ee None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:35.354467 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7a209a6e-6d72-4e1e-ada2-f88826b4e8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:35.419382 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c3715-cf58-474b-ac89-6484b0bc54c8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:35.435168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c2378-f77d-4e26-b71f-8a5d26e56176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:35.435870 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c2378-f77d-4e26-b71f-8a5d26e56176 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:35.436186 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c2378-f77d-4e26-b71f-8a5d26e56176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:35.436468 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c2378-f77d-4e26-b71f-8a5d26e56176 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:35.437182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c2378-f77d-4e26-b71f-8a5d26e56176 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:35.439559 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c2378-f77d-4e26-b71f-8a5d26e56176 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:35.483868 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 838/1660] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:08:35 2022] POST /identity/v3/users => generated 291 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:35.488092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c2378-f77d-4e26-b71f-8a5d26e56176 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:35.489523 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 823/1661] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:35.493853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a209a6e-6d72-4e1e-ada2-f88826b4e8ee None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:35.514739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c2378-f77d-4e26-b71f-8a5d26e56176 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:35.516316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a209a6e-6d72-4e1e-ada2-f88826b4e8ee None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:35.521662 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf0c64-7d3c-42b6-9e71-c5a5877713c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:35.522473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf0c64-7d3c-42b6-9e71-c5a5877713c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:35.522880 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf0c64-7d3c-42b6-9e71-c5a5877713c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:35.523295 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf0c64-7d3c-42b6-9e71-c5a5877713c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:35.524176 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf0c64-7d3c-42b6-9e71-c5a5877713c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:35.525111 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf0c64-7d3c-42b6-9e71-c5a5877713c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:35.528762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888af850-2f31-43ca-bf15-069b4a09ce35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:35.529920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888af850-2f31-43ca-bf15-069b4a09ce35 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:35.529920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888af850-2f31-43ca-bf15-069b4a09ce35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:35.530093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888af850-2f31-43ca-bf15-069b4a09ce35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:35.530375 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf0c64-7d3c-42b6-9e71-c5a5877713c8 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:35.530811 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888af850-2f31-43ca-bf15-069b4a09ce35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:35.532978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888af850-2f31-43ca-bf15-069b4a09ce35 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:35.562571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 824/1662] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:08:35.567695 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf0c64-7d3c-42b6-9e71-c5a5877713c8 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:35.579267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 839/1663] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:35.583497 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888af850-2f31-43ca-bf15-069b4a09ce35 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:35.584516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf0c64-7d3c-42b6-9e71-c5a5877713c8 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:35.595983 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4a9c3-e654-4566-9c84-2661a27077f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:35.596658 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4a9c3-e654-4566-9c84-2661a27077f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:35.597318 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4a9c3-e654-4566-9c84-2661a27077f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:35.597736 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4a9c3-e654-4566-9c84-2661a27077f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:35.598658 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4a9c3-e654-4566-9c84-2661a27077f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:35.600780 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888af850-2f31-43ca-bf15-069b4a09ce35 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:35.600991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4a9c3-e654-4566-9c84-2661a27077f5 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:35.607561 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3e4a8-2b06-454e-b433-7019af55204c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:35.608618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3e4a8-2b06-454e-b433-7019af55204c admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:35.609129 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3e4a8-2b06-454e-b433-7019af55204c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:35.609521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3e4a8-2b06-454e-b433-7019af55204c admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:35.610528 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3e4a8-2b06-454e-b433-7019af55204c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:35.612690 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3e4a8-2b06-454e-b433-7019af55204c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:35.642925 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 840/1664] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:35.644446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3e4a8-2b06-454e-b433-7019af55204c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:35.651997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:35.660454 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3e4a8-2b06-454e-b433-7019af55204c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:35.663147 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 825/1665] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:08:35 2022] POST /identity/v3/projects => generated 367 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:08:35.668151 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4a9c3-e654-4566-9c84-2661a27077f5 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:35.782587 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366435a-9d7b-4074-b91e-b510ac343226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:35.783677 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366435a-9d7b-4074-b91e-b510ac343226 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:35.785076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366435a-9d7b-4074-b91e-b510ac343226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:35.786668 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366435a-9d7b-4074-b91e-b510ac343226 admin admin] PATH_INFO: `/v3/projects/9d7c0a3be6034542891b016964d8d70e/users/c552978160c8455eb6d2986c5bfc9b01/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:35.831048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366435a-9d7b-4074-b91e-b510ac343226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d7c0a3be6034542891b016964d8d70e, user_id=c552978160c8455eb6d2986c5bfc9b01, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:35.840144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4a9c3-e654-4566-9c84-2661a27077f5 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:35.841059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366435a-9d7b-4074-b91e-b510ac343226 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:35.855037 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e1ac1b-7b14-4ccb-ae3d-f1e04b997534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:35.855774 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e1ac1b-7b14-4ccb-ae3d-f1e04b997534 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:35.856120 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e1ac1b-7b14-4ccb-ae3d-f1e04b997534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:35.856408 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e1ac1b-7b14-4ccb-ae3d-f1e04b997534 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:35.857128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e1ac1b-7b14-4ccb-ae3d-f1e04b997534 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:35.859220 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e1ac1b-7b14-4ccb-ae3d-f1e04b997534 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:35.886250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 841/1666] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:08:35 2022] PUT /identity/v3/projects/9d7c0a3be6034542891b016964d8d70e/users/c552978160c8455eb6d2986c5bfc9b01/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:35.889697 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba943ff-1701-446a-94cc-dd537987e076 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:35.891237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba943ff-1701-446a-94cc-dd537987e076 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:35.892222 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba943ff-1701-446a-94cc-dd537987e076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:35.906778 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 826/1667] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:08:35 2022] POST /identity/v3/users => generated 284 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:35.910947 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e1ac1b-7b14-4ccb-ae3d-f1e04b997534 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:35.932693 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e1ac1b-7b14-4ccb-ae3d-f1e04b997534 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:35.966879 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4ba943ff-1701-446a-94cc-dd537987e076 None None] MFA Rules not processed for user `c552978160c8455eb6d2986c5bfc9b01`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:35.968920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4ba943ff-1701-446a-94cc-dd537987e076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:36.091557 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:36.107823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fe60c-0880-4be8-a882-41ad08d8754a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:36.110349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fe60c-0880-4be8-a882-41ad08d8754a admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:36.110750 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fe60c-0880-4be8-a882-41ad08d8754a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:36.112742 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1fe60c-0880-4be8-a882-41ad08d8754a admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:36.113925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fe60c-0880-4be8-a882-41ad08d8754a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:36.122489 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1fe60c-0880-4be8-a882-41ad08d8754a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:36.139896 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 827/1668] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:08:36.144232 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fe60c-0880-4be8-a882-41ad08d8754a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:36.160934 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1fe60c-0880-4be8-a882-41ad08d8754a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:36.167909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6df2ef-4601-4ae9-b675-4439435719e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:36.171905 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6df2ef-4601-4ae9-b675-4439435719e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:36.172501 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6df2ef-4601-4ae9-b675-4439435719e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:36.173111 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6df2ef-4601-4ae9-b675-4439435719e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:36.173694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 842/1669] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:36.174442 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6df2ef-4601-4ae9-b675-4439435719e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:36.178363 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6df2ef-4601-4ae9-b675-4439435719e2 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:36.178583 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba943ff-1701-446a-94cc-dd537987e076 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:36.196791 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 828/1670] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:36.197983 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba943ff-1701-446a-94cc-dd537987e076 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:36.200960 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6df2ef-4601-4ae9-b675-4439435719e2 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:36.213357 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6df2ef-4601-4ae9-b675-4439435719e2 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:36.215666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6643a3-a118-4417-8285-dc2f659bfbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:36.216408 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6643a3-a118-4417-8285-dc2f659bfbab admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:36.216703 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6643a3-a118-4417-8285-dc2f659bfbab admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:36.216985 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6643a3-a118-4417-8285-dc2f659bfbab admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:36.217694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6643a3-a118-4417-8285-dc2f659bfbab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:36.219628 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6643a3-a118-4417-8285-dc2f659bfbab admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:36.219781 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b1904-9c61-4a62-a296-91bbc933efb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:36.220769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b1904-9c61-4a62-a296-91bbc933efb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:36.221165 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b1904-9c61-4a62-a296-91bbc933efb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:36.221569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b1904-9c61-4a62-a296-91bbc933efb3 admin admin] PATH_INFO: `/v3/projects/4a1a537bce174a47aeb560b45fd0d738/users/75e118c1d0fd4c0fa95490bf6d46f926/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:36.258394 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b1904-9c61-4a62-a296-91bbc933efb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a1a537bce174a47aeb560b45fd0d738, user_id=75e118c1d0fd4c0fa95490bf6d46f926, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:36.262773 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b1904-9c61-4a62-a296-91bbc933efb3 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:36.281610 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 843/1671] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:08:36 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:08:36.283536 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6643a3-a118-4417-8285-dc2f659bfbab admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:36.298721 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 829/1672] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:08:36 2022] PUT /identity/v3/projects/4a1a537bce174a47aeb560b45fd0d738/users/75e118c1d0fd4c0fa95490bf6d46f926/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:36.300848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5b619-4dd0-4721-9489-6d1c4abbd05c None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:36.301179 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5b619-4dd0-4721-9489-6d1c4abbd05c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:36.302199 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5b619-4dd0-4721-9489-6d1c4abbd05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:36.374397 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:36.383439 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-81e5b619-4dd0-4721-9489-6d1c4abbd05c None None] MFA Rules not processed for user `75e118c1d0fd4c0fa95490bf6d46f926`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:36.392382 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-81e5b619-4dd0-4721-9489-6d1c4abbd05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:36.396024 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6643a3-a118-4417-8285-dc2f659bfbab admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:36.517968 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf152a6-fe71-4b5e-8446-6afde9697549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:36.519102 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf152a6-fe71-4b5e-8446-6afde9697549 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:36.519753 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf152a6-fe71-4b5e-8446-6afde9697549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:36.520242 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf152a6-fe71-4b5e-8446-6afde9697549 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:36.521127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf152a6-fe71-4b5e-8446-6afde9697549 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:36.523817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf152a6-fe71-4b5e-8446-6afde9697549 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:36.557488 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 844/1673] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:08:36 2022] POST /identity/v3/users => generated 297 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:36.557488 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b6eff-f2ba-41dd-93fe-12e67d14bee5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:36.557488 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b6eff-f2ba-41dd-93fe-12e67d14bee5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:36.557488 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b6eff-f2ba-41dd-93fe-12e67d14bee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:36.579703 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 830/1674] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:36 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:36.579703 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5b619-4dd0-4721-9489-6d1c4abbd05c None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:36.597968 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-876b6eff-f2ba-41dd-93fe-12e67d14bee5 None None] MFA Rules not processed for user `0542cd47ba77454ba25170292ed9aeb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:36.599276 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5b619-4dd0-4721-9489-6d1c4abbd05c None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:36.601064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-876b6eff-f2ba-41dd-93fe-12e67d14bee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:36.606521 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd0d89-75b0-43e2-896f-3b2d9eeda343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:36.607538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd0d89-75b0-43e2-896f-3b2d9eeda343 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:36.608172 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd0d89-75b0-43e2-896f-3b2d9eeda343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:36.608793 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd0d89-75b0-43e2-896f-3b2d9eeda343 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:36.610191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd0d89-75b0-43e2-896f-3b2d9eeda343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:36.612234 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd0d89-75b0-43e2-896f-3b2d9eeda343 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:36.625683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 831/1675] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:36.626834 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd0d89-75b0-43e2-896f-3b2d9eeda343 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:36.635194 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d2bd0d89-75b0-43e2-896f-3b2d9eeda343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:36.758954 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 845/1676] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:36.759583 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b6eff-f2ba-41dd-93fe-12e67d14bee5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:36.779618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b6eff-f2ba-41dd-93fe-12e67d14bee5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:36.793322 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd0d89-75b0-43e2-896f-3b2d9eeda343 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:36.803734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf197bae-5a1d-4d1e-be38-7bdabba14563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:36.804909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037a9cc-49af-4b9c-ac55-60c5c44f1c4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:36.805137 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf197bae-5a1d-4d1e-be38-7bdabba14563 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:36.805682 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf197bae-5a1d-4d1e-be38-7bdabba14563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:36.805803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037a9cc-49af-4b9c-ac55-60c5c44f1c4e service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:36.806093 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037a9cc-49af-4b9c-ac55-60c5c44f1c4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:36.806265 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037a9cc-49af-4b9c-ac55-60c5c44f1c4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:36.806728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf197bae-5a1d-4d1e-be38-7bdabba14563 admin admin] PATH_INFO: `/v3/projects/3ed1174b5a5848bca12ad18c4fd6447c/users/4bac5d3327f847d9836a86ca97b1b3cf/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:36.829610 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9037a9cc-49af-4b9c-ac55-60c5c44f1c4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:36.831308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9037a9cc-49af-4b9c-ac55-60c5c44f1c4e service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:36.844835 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf197bae-5a1d-4d1e-be38-7bdabba14563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ed1174b5a5848bca12ad18c4fd6447c, user_id=4bac5d3327f847d9836a86ca97b1b3cf, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:36.849895 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf197bae-5a1d-4d1e-be38-7bdabba14563 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:36.858319 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 832/1677] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:36 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:36.859282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037a9cc-49af-4b9c-ac55-60c5c44f1c4e service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:36.868102 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9037a9cc-49af-4b9c-ac55-60c5c44f1c4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:36.872498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 846/1678] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:08:36 2022] PUT /identity/v3/projects/3ed1174b5a5848bca12ad18c4fd6447c/users/4bac5d3327f847d9836a86ca97b1b3cf/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:36.877212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf197bae-5a1d-4d1e-be38-7bdabba14563 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:36.974284 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:36.977010 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:36.987057 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf197bae-5a1d-4d1e-be38-7bdabba14563 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:36.998730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048896f-4e80-4464-a52e-be5f02952a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:37.000650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048896f-4e80-4464-a52e-be5f02952a00 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:37.001442 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048896f-4e80-4464-a52e-be5f02952a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:37.002230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048896f-4e80-4464-a52e-be5f02952a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:37.003090 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048896f-4e80-4464-a52e-be5f02952a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:37.008037 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048896f-4e80-4464-a52e-be5f02952a00 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:37.032702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 847/1679] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:37.035307 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048896f-4e80-4464-a52e-be5f02952a00 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:37.044632 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037a9cc-49af-4b9c-ac55-60c5c44f1c4e service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:37.055014 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048896f-4e80-4464-a52e-be5f02952a00 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:37.068105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b94d37d-d85c-4a09-81f5-8c678f4aa7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:37.074046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94d37d-d85c-4a09-81f5-8c678f4aa7fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:37.074480 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94d37d-d85c-4a09-81f5-8c678f4aa7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:37.074879 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94d37d-d85c-4a09-81f5-8c678f4aa7fb admin admin] PATH_INFO: `/v3/projects/3ed1174b5a5848bca12ad18c4fd6447c/users/4bac5d3327f847d9836a86ca97b1b3cf/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:37.078406 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b94d37d-d85c-4a09-81f5-8c678f4aa7fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ed1174b5a5848bca12ad18c4fd6447c, user_id=4bac5d3327f847d9836a86ca97b1b3cf, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:37.081497 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b94d37d-d85c-4a09-81f5-8c678f4aa7fb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:37.107987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 848/1680] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:08:37 2022] PUT /identity/v3/projects/3ed1174b5a5848bca12ad18c4fd6447c/users/4bac5d3327f847d9836a86ca97b1b3cf/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:37.119313 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbc362-e72b-431a-b7a9-d8b8d4947d11 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:37.119970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbc362-e72b-431a-b7a9-d8b8d4947d11 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:37.120492 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbc362-e72b-431a-b7a9-d8b8d4947d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:37.200138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4adbc362-e72b-431a-b7a9-d8b8d4947d11 None None] MFA Rules not processed for user `4bac5d3327f847d9836a86ca97b1b3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:37.201405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4adbc362-e72b-431a-b7a9-d8b8d4947d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:37.209313 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6764be-d94a-4255-a31b-8e1f9f9a72ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:37.209790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6764be-d94a-4255-a31b-8e1f9f9a72ac service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:37.210438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6764be-d94a-4255-a31b-8e1f9f9a72ac service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:37.210438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6764be-d94a-4255-a31b-8e1f9f9a72ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:37.390942 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6764be-d94a-4255-a31b-8e1f9f9a72ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:37.397338 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6764be-d94a-4255-a31b-8e1f9f9a72ac service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:37.436590 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 849/1681] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:37 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:37.453046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adbc362-e72b-431a-b7a9-d8b8d4947d11 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:37.457783 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 833/1682] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:37.660031 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adbc362-e72b-431a-b7a9-d8b8d4947d11 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:37.673150 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6203a4-64c8-416c-be68-c9f8ae5bf261 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:37.674685 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6203a4-64c8-416c-be68-c9f8ae5bf261 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:37.675123 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6203a4-64c8-416c-be68-c9f8ae5bf261 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:37.675736 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6203a4-64c8-416c-be68-c9f8ae5bf261 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:37.696430 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6203a4-64c8-416c-be68-c9f8ae5bf261 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:37.698789 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6203a4-64c8-416c-be68-c9f8ae5bf261 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:37.719256 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 850/1683] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:37.890892 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6764be-d94a-4255-a31b-8e1f9f9a72ac service nova] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:38.043532 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6764be-d94a-4255-a31b-8e1f9f9a72ac service nova] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:38.052896 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76db535-c330-49a0-8e5d-02287bf2a43b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:38.054053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76db535-c330-49a0-8e5d-02287bf2a43b service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:38.055875 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76db535-c330-49a0-8e5d-02287bf2a43b service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:38.056457 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76db535-c330-49a0-8e5d-02287bf2a43b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:38.112208 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:38.168958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76db535-c330-49a0-8e5d-02287bf2a43b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:38.172293 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76db535-c330-49a0-8e5d-02287bf2a43b service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:38.189529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 834/1684] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:41.091777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9a61b-bf7e-4acc-9795-9e4ddace2df0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:41.092334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9a61b-bf7e-4acc-9795-9e4ddace2df0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:41.092439 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9a61b-bf7e-4acc-9795-9e4ddace2df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:41.111987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-41d9a61b-bf7e-4acc-9795-9e4ddace2df0 None None] MFA Rules not processed for user `0542cd47ba77454ba25170292ed9aeb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:41.113205 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-41d9a61b-bf7e-4acc-9795-9e4ddace2df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:41.215385 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 851/1685] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:41.437475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca6ef7-056a-4a5a-84b4-e37db9803aaf None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:41.437968 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca6ef7-056a-4a5a-84b4-e37db9803aaf None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:41.438382 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca6ef7-056a-4a5a-84b4-e37db9803aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:41.515850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-e7ca6ef7-056a-4a5a-84b4-e37db9803aaf None None] MFA Rules not processed for user `226d34540e464a93aa984ff59b497450`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:41.517068 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e7ca6ef7-056a-4a5a-84b4-e37db9803aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:41.644419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 835/1686] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:41.656767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1280149-5282-4c5c-824c-c3a7eb3da455 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:41.658859 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1280149-5282-4c5c-824c-c3a7eb3da455 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:41.659268 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1280149-5282-4c5c-824c-c3a7eb3da455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:41.689986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e1280149-5282-4c5c-824c-c3a7eb3da455 None None] MFA Rules not processed for user `226d34540e464a93aa984ff59b497450`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:41.692617 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e1280149-5282-4c5c-824c-c3a7eb3da455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:41.708675 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 852/1687] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:41.712854 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca6ef7-056a-4a5a-84b4-e37db9803aaf None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:41.715066 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e7ca6ef7-056a-4a5a-84b4-e37db9803aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:41.856748 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca6ef7-056a-4a5a-84b4-e37db9803aaf None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:41.901334 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21afc376-beb3-4e34-a27e-0a96ac30e14e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:41.903742 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afc376-beb3-4e34-a27e-0a96ac30e14e service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:41.905089 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afc376-beb3-4e34-a27e-0a96ac30e14e service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:41.906202 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afc376-beb3-4e34-a27e-0a96ac30e14e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:41.920637 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21afc376-beb3-4e34-a27e-0a96ac30e14e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:41.926208 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21afc376-beb3-4e34-a27e-0a96ac30e14e service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:41.960735 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 836/1688] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:08:42.145455 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c055a-f15e-493a-b1b5-8beb27bf69ec None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:42.146639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c055a-f15e-493a-b1b5-8beb27bf69ec None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:42.146951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c055a-f15e-493a-b1b5-8beb27bf69ec None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:42.150034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 853/1689] 10.222.0.94 () {54 vars in 912 bytes} [Mon Jul 18 14:08:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:08:42.165643 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ed932-0633-41ee-82df-ea6f5af02b29 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:42.165643 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ed932-0633-41ee-82df-ea6f5af02b29 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:42.165643 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ed932-0633-41ee-82df-ea6f5af02b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:42.222739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-4d1ed932-0633-41ee-82df-ea6f5af02b29 None None] MFA Rules not processed for user `14027a0e13fd4272bd7803e3131b6348`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:42.224576 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4d1ed932-0633-41ee-82df-ea6f5af02b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:42.264173 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:42.298527 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 837/1690] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Jul 18 14:08:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:42.320505 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c055a-f15e-493a-b1b5-8beb27bf69ec None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:42.323174 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b20c055a-f15e-493a-b1b5-8beb27bf69ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:42.352226 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924f7e0-461b-4ecd-8bdf-93525bbb7413 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:42.352517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924f7e0-461b-4ecd-8bdf-93525bbb7413 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:42.352801 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924f7e0-461b-4ecd-8bdf-93525bbb7413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:42.360323 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:42.372578 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c055a-f15e-493a-b1b5-8beb27bf69ec None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:42.378094 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3afdac6-0b32-47f9-a24e-00cfcbb2c0c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:42.379047 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afdac6-0b32-47f9-a24e-00cfcbb2c0c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:42.379538 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afdac6-0b32-47f9-a24e-00cfcbb2c0c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:42.380077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afdac6-0b32-47f9-a24e-00cfcbb2c0c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:42.390876 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3afdac6-0b32-47f9-a24e-00cfcbb2c0c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:42.393011 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3afdac6-0b32-47f9-a24e-00cfcbb2c0c8 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:42.406259 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 854/1691] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:42.408864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0a392-244b-4b86-abc5-d58dc604f1f8 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:42.409400 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0a392-244b-4b86-abc5-d58dc604f1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:42.409700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d924f7e0-461b-4ecd-8bdf-93525bbb7413 None None] MFA Rules not processed for user `3720c2225b51464abee34d05a69b9db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:42.410107 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0a392-244b-4b86-abc5-d58dc604f1f8 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:42.412242 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 855/1692] 10.222.0.94 () {54 vars in 912 bytes} [Mon Jul 18 14:08:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:08:42.415164 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d924f7e0-461b-4ecd-8bdf-93525bbb7413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:42.420191 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb10ef-a128-41d5-b0ec-a4d4277e8839 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:42.420792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb10ef-a128-41d5-b0ec-a4d4277e8839 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:42.421367 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb10ef-a128-41d5-b0ec-a4d4277e8839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:42.467889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-85cb10ef-a128-41d5-b0ec-a4d4277e8839 None None] MFA Rules not processed for user `14027a0e13fd4272bd7803e3131b6348`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:42.472558 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-85cb10ef-a128-41d5-b0ec-a4d4277e8839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:42.493629 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 856/1693] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Jul 18 14:08:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:08:42.540310 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb10ef-a128-41d5-b0ec-a4d4277e8839 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:42.550480 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 838/1694] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:42.552843 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb10ef-a128-41d5-b0ec-a4d4277e8839 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:42.565446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666dd9e9-25f9-4267-8b4d-e5956098d8f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:42.566503 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dd9e9-25f9-4267-8b4d-e5956098d8f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:42.569002 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dd9e9-25f9-4267-8b4d-e5956098d8f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:42.569247 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a35f1-1c9d-43e6-8866-414ed7805e77 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:42.569553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a35f1-1c9d-43e6-8866-414ed7805e77 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:42.569842 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a35f1-1c9d-43e6-8866-414ed7805e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:42.570248 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666dd9e9-25f9-4267-8b4d-e5956098d8f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:42.597496 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666dd9e9-25f9-4267-8b4d-e5956098d8f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:42.600871 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666dd9e9-25f9-4267-8b4d-e5956098d8f5 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:42.621755 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 857/1695] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:42.623475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-658a35f1-1c9d-43e6-8866-414ed7805e77 None None] MFA Rules not processed for user `3720c2225b51464abee34d05a69b9db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:42.624761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-658a35f1-1c9d-43e6-8866-414ed7805e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:42.649276 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 839/1696] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:42.709942 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840d387-1460-4de4-abd3-7a369f782284 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:42.710360 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840d387-1460-4de4-abd3-7a369f782284 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:42.710749 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840d387-1460-4de4-abd3-7a369f782284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:42.751734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b840d387-1460-4de4-abd3-7a369f782284 None None] MFA Rules not processed for user `0b6cb92018ea4cc5a25c5423b9bcd7a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:42.753096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b840d387-1460-4de4-abd3-7a369f782284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:42.872013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 858/1697] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:43.347869 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfa227-5223-4d78-88a8-75f1842c97b2 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:43.348199 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfa227-5223-4d78-88a8-75f1842c97b2 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:43.348493 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfa227-5223-4d78-88a8-75f1842c97b2 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:43.350558 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 840/1698] 10.222.0.94 () {54 vars in 961 bytes} [Mon Jul 18 14:08:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:08:43.357482 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e94c4b-2ee9-4eb7-a335-b8bfed1ba265 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:43.357939 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e94c4b-2ee9-4eb7-a335-b8bfed1ba265 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:43.358167 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e94c4b-2ee9-4eb7-a335-b8bfed1ba265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:43.384172 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebfa227-5223-4d78-88a8-75f1842c97b2 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:43.421154 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d0e94c4b-2ee9-4eb7-a335-b8bfed1ba265 None None] MFA Rules not processed for user `dd5b2ba9e37041c78a99f1700a4970c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:43.422078 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebfa227-5223-4d78-88a8-75f1842c97b2 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:43.426083 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d0e94c4b-2ee9-4eb7-a335-b8bfed1ba265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:43.468093 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee55b5f-635e-42bb-a963-3f8a53aa7a97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:43.468770 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee55b5f-635e-42bb-a963-3f8a53aa7a97 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:43.469062 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee55b5f-635e-42bb-a963-3f8a53aa7a97 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:43.469388 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee55b5f-635e-42bb-a963-3f8a53aa7a97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:43.504715 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee55b5f-635e-42bb-a963-3f8a53aa7a97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:43.509385 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee55b5f-635e-42bb-a963-3f8a53aa7a97 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:43.527141 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 841/1699] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:43.528190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee55b5f-635e-42bb-a963-3f8a53aa7a97 service nova] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:43.559529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee55b5f-635e-42bb-a963-3f8a53aa7a97 service nova] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:43.595966 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60a3f7-8df5-4217-b89c-eb2555a9c866 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:43.597050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60a3f7-8df5-4217-b89c-eb2555a9c866 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:43.598354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60a3f7-8df5-4217-b89c-eb2555a9c866 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:43.598835 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60a3f7-8df5-4217-b89c-eb2555a9c866 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:43.631046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 859/1700] 10.222.0.94 () {58 vars in 1058 bytes} [Mon Jul 18 14:08:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:43.643440 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60a3f7-8df5-4217-b89c-eb2555a9c866 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:43.643651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e94c4b-2ee9-4eb7-a335-b8bfed1ba265 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:43.644927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d0e94c4b-2ee9-4eb7-a335-b8bfed1ba265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:43.645128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60a3f7-8df5-4217-b89c-eb2555a9c866 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:43.663719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e94c4b-2ee9-4eb7-a335-b8bfed1ba265 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:43.670654 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 842/1701] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:43.671761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60a3f7-8df5-4217-b89c-eb2555a9c866 service nova] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:43.676404 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43325984-9474-495e-ba63-e2f8e5ae82af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd5b2ba9e37041c78a99f1700a4970c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:43.678012 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43325984-9474-495e-ba63-e2f8e5ae82af service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:43.678310 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43325984-9474-495e-ba63-e2f8e5ae82af service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:43.678597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43325984-9474-495e-ba63-e2f8e5ae82af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:43.681340 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-43325984-9474-495e-ba63-e2f8e5ae82af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:43.689068 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60a3f7-8df5-4217-b89c-eb2555a9c866 service nova] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:43.696222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74018a19-d800-425e-a5a1-7e08065c7539 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd5b2ba9e37041c78a99f1700a4970c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:43.697241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74018a19-d800-425e-a5a1-7e08065c7539 service placement] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:43.697813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74018a19-d800-425e-a5a1-7e08065c7539 service placement] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:43.698296 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74018a19-d800-425e-a5a1-7e08065c7539 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:43.700826 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-74018a19-d800-425e-a5a1-7e08065c7539 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:43.717442 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74018a19-d800-425e-a5a1-7e08065c7539 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:43.719401 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74018a19-d800-425e-a5a1-7e08065c7539 service placement] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:43.728955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43325984-9474-495e-ba63-e2f8e5ae82af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:43.735880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43325984-9474-495e-ba63-e2f8e5ae82af service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:43.743206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 843/1702] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:43.776696 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 860/1703] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:45.828051 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74018a19-d800-425e-a5a1-7e08065c7539 service placement] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:45.830691 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-74018a19-d800-425e-a5a1-7e08065c7539 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:46.013525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43325984-9474-495e-ba63-e2f8e5ae82af service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:46.167079 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74018a19-d800-425e-a5a1-7e08065c7539 service placement] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:46.181074 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebb87b-085c-495b-8237-7ce8767aabfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:46.183763 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebb87b-085c-495b-8237-7ce8767aabfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:46.189633 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebb87b-085c-495b-8237-7ce8767aabfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:46.190080 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebb87b-085c-495b-8237-7ce8767aabfc admin admin] PATH_INFO: `/v3/users/d3e29b6b60a848d59e23128b1ada1d9d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:46.196606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ebb87b-085c-495b-8237-7ce8767aabfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3e29b6b60a848d59e23128b1ada1d9d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:46.203554 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ebb87b-085c-495b-8237-7ce8767aabfc admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:46.218319 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:46.252375 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43325984-9474-495e-ba63-e2f8e5ae82af service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:46.260223 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53523299-4ddb-427b-a0e2-5e91ac26a8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:46.261236 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53523299-4ddb-427b-a0e2-5e91ac26a8e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:46.261543 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53523299-4ddb-427b-a0e2-5e91ac26a8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:46.261832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53523299-4ddb-427b-a0e2-5e91ac26a8e1 admin admin] PATH_INFO: `/v3/users/c552978160c8455eb6d2986c5bfc9b01` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:46.264488 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53523299-4ddb-427b-a0e2-5e91ac26a8e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c552978160c8455eb6d2986c5bfc9b01)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:46.266501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53523299-4ddb-427b-a0e2-5e91ac26a8e1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:46.310090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:46.362830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-53523299-4ddb-427b-a0e2-5e91ac26a8e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c552978160c8455eb6d2986c5bfc9b01'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:46.375629 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-53523299-4ddb-427b-a0e2-5e91ac26a8e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c552978160c8455eb6d2986c5bfc9b01'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:46.387856 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d2ebb87b-085c-495b-8237-7ce8767aabfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3e29b6b60a848d59e23128b1ada1d9d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:46.394456 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-53523299-4ddb-427b-a0e2-5e91ac26a8e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c552978160c8455eb6d2986c5bfc9b01'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:46.396850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-53523299-4ddb-427b-a0e2-5e91ac26a8e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c552978160c8455eb6d2986c5bfc9b01'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:46.408324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d2ebb87b-085c-495b-8237-7ce8767aabfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3e29b6b60a848d59e23128b1ada1d9d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:46.417463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-53523299-4ddb-427b-a0e2-5e91ac26a8e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c552978160c8455eb6d2986c5bfc9b01'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:46.440121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d2ebb87b-085c-495b-8237-7ce8767aabfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3e29b6b60a848d59e23128b1ada1d9d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:46.441052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d2ebb87b-085c-495b-8237-7ce8767aabfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3e29b6b60a848d59e23128b1ada1d9d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:46.449484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 861/1704] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:08:46 2022] DELETE /identity/v3/users/c552978160c8455eb6d2986c5bfc9b01 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:08:46.451008 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4acd3b-980f-4a0f-ab58-953b53d45de9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:46.451246 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4acd3b-980f-4a0f-ab58-953b53d45de9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:46.451532 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4acd3b-980f-4a0f-ab58-953b53d45de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:46.510348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d2ebb87b-085c-495b-8237-7ce8767aabfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3e29b6b60a848d59e23128b1ada1d9d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:46.515854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3b4acd3b-980f-4a0f-ab58-953b53d45de9 None None] MFA Rules not processed for user `3720c2225b51464abee34d05a69b9db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:46.517329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3b4acd3b-980f-4a0f-ab58-953b53d45de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:46.544149 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 844/1705] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:08:45 2022] DELETE /identity/v3/users/d3e29b6b60a848d59e23128b1ada1d9d => generated 0 bytes in 718 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:46.546456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39294cc6-cdec-4ba3-94ac-02c26941edfd None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:46.552368 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39294cc6-cdec-4ba3-94ac-02c26941edfd None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:46.554352 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39294cc6-cdec-4ba3-94ac-02c26941edfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:46.600722 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-39294cc6-cdec-4ba3-94ac-02c26941edfd None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:46.600722 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-39294cc6-cdec-4ba3-94ac-02c26941edfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:46.683888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 862/1706] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:46.688999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b7fcd-e45a-4b16-984e-82d3cd815cdc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:46.689320 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b7fcd-e45a-4b16-984e-82d3cd815cdc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:46.689612 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b7fcd-e45a-4b16-984e-82d3cd815cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:46.763173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c78b7fcd-e45a-4b16-984e-82d3cd815cdc None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:46.764738 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c78b7fcd-e45a-4b16-984e-82d3cd815cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:46.784963 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 845/1707] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:46.792029 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39294cc6-cdec-4ba3-94ac-02c26941edfd None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:46.797641 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-39294cc6-cdec-4ba3-94ac-02c26941edfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:46.960849 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 863/1708] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:46.961567 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b7fcd-e45a-4b16-984e-82d3cd815cdc None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:46.965479 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c78b7fcd-e45a-4b16-984e-82d3cd815cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:46.997056 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39294cc6-cdec-4ba3-94ac-02c26941edfd None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:46.999689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b7fcd-e45a-4b16-984e-82d3cd815cdc None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:47.014390 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0636e9-3030-4ecb-ab37-45706c022a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:47.016304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0636e9-3030-4ecb-ab37-45706c022a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:47.018139 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b421de-55f9-4869-8edd-603ab5c5010a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:47.018569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0636e9-3030-4ecb-ab37-45706c022a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:47.019188 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b421de-55f9-4869-8edd-603ab5c5010a service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:47.019337 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0636e9-3030-4ecb-ab37-45706c022a1d admin admin] PATH_INFO: `/v3/users/90d14aedc7b343e2996b7295b3733d2c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:47.019496 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b421de-55f9-4869-8edd-603ab5c5010a service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:47.019933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b421de-55f9-4869-8edd-603ab5c5010a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:47.022610 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0636e9-3030-4ecb-ab37-45706c022a1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=90d14aedc7b343e2996b7295b3733d2c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:47.031579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0636e9-3030-4ecb-ab37-45706c022a1d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:47.045720 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:47.046852 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b421de-55f9-4869-8edd-603ab5c5010a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:47.052271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b421de-55f9-4869-8edd-603ab5c5010a service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:47.086507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 846/1709] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:47.092001 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b421de-55f9-4869-8edd-603ab5c5010a service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:47.097212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e5b421de-55f9-4869-8edd-603ab5c5010a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:47.124276 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b421de-55f9-4869-8edd-603ab5c5010a service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:47.142034 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676b431-207b-47d3-bb38-fa4de1060491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:47.142996 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676b431-207b-47d3-bb38-fa4de1060491 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:47.143436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676b431-207b-47d3-bb38-fa4de1060491 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:47.143872 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676b431-207b-47d3-bb38-fa4de1060491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:47.155276 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2b0636e9-3030-4ecb-ab37-45706c022a1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90d14aedc7b343e2996b7295b3733d2c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:47.164807 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6676b431-207b-47d3-bb38-fa4de1060491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:47.166530 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6676b431-207b-47d3-bb38-fa4de1060491 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:47.173636 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2b0636e9-3030-4ecb-ab37-45706c022a1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90d14aedc7b343e2996b7295b3733d2c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:47.202219 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 847/1710] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:47.220254 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2b0636e9-3030-4ecb-ab37-45706c022a1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90d14aedc7b343e2996b7295b3733d2c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:47.223663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2b0636e9-3030-4ecb-ab37-45706c022a1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90d14aedc7b343e2996b7295b3733d2c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:47.263116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2b0636e9-3030-4ecb-ab37-45706c022a1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90d14aedc7b343e2996b7295b3733d2c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:47.279869 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefdc5d1-231a-41ce-b73a-ee60e23d21de None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:47.282961 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefdc5d1-231a-41ce-b73a-ee60e23d21de None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:47.283470 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefdc5d1-231a-41ce-b73a-ee60e23d21de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:47.284451 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 864/1711] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:08:46 2022] DELETE /identity/v3/users/90d14aedc7b343e2996b7295b3733d2c => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:47.329984 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-fefdc5d1-231a-41ce-b73a-ee60e23d21de None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:47.333970 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fefdc5d1-231a-41ce-b73a-ee60e23d21de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:47.403111 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:47.490766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 848/1712] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:47.535237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0636e9-3030-4ecb-ab37-45706c022a1d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:47.537020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2b0636e9-3030-4ecb-ab37-45706c022a1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:47.713646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0636e9-3030-4ecb-ab37-45706c022a1d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:47.746324 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17990e2f-e741-494a-82ed-d3042f541ca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:47.747075 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17990e2f-e741-494a-82ed-d3042f541ca1 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:47.749057 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17990e2f-e741-494a-82ed-d3042f541ca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:47.749352 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17990e2f-e741-494a-82ed-d3042f541ca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:47.779075 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17990e2f-e741-494a-82ed-d3042f541ca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:47.783564 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17990e2f-e741-494a-82ed-d3042f541ca1 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:47.826960 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 865/1713] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:48.483873 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefdc5d1-231a-41ce-b73a-ee60e23d21de None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:48.485416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fefdc5d1-231a-41ce-b73a-ee60e23d21de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:48.517541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefdc5d1-231a-41ce-b73a-ee60e23d21de None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:48.564844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07ced8-0ffa-487e-b91f-9319231afc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:48.566375 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07ced8-0ffa-487e-b91f-9319231afc2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:48.566959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07ced8-0ffa-487e-b91f-9319231afc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:48.567248 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07ced8-0ffa-487e-b91f-9319231afc2d admin admin] PATH_INFO: `/v3/projects/c9a6664b4d194755bc4b8e46ef187a7a` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:48.575027 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a07ced8-0ffa-487e-b91f-9319231afc2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9a6664b4d194755bc4b8e46ef187a7a)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:48.578534 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a07ced8-0ffa-487e-b91f-9319231afc2d admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:48.697421 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9124dd3-3ec0-48b6-9fe4-2eb371d80b51 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:48.697977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9124dd3-3ec0-48b6-9fe4-2eb371d80b51 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:48.697977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9124dd3-3ec0-48b6-9fe4-2eb371d80b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:48.804678 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4a07ced8-0ffa-487e-b91f-9319231afc2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9a6664b4d194755bc4b8e46ef187a7a', 'request_id': 'req-4a07ced8-0ffa-487e-b91f-9319231afc2d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:48.821962 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4a07ced8-0ffa-487e-b91f-9319231afc2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9a6664b4d194755bc4b8e46ef187a7a', 'request_id': 'req-4a07ced8-0ffa-487e-b91f-9319231afc2d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:48.839455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4a07ced8-0ffa-487e-b91f-9319231afc2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9a6664b4d194755bc4b8e46ef187a7a', 'request_id': 'req-4a07ced8-0ffa-487e-b91f-9319231afc2d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:48.882881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4a07ced8-0ffa-487e-b91f-9319231afc2d admin admin] The token cache is being invalidate because project c9a6664b4d194755bc4b8e46ef187a7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:08:48.883180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4a07ced8-0ffa-487e-b91f-9319231afc2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:48.884762 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4a07ced8-0ffa-487e-b91f-9319231afc2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:48.885985 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c9124dd3-3ec0-48b6-9fe4-2eb371d80b51 None None] MFA Rules not processed for user `94d824bffc2c4af299c537056bc21734`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:48.887247 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c9124dd3-3ec0-48b6-9fe4-2eb371d80b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:48.894240 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 849/1714] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:08:48 2022] DELETE /identity/v3/projects/c9a6664b4d194755bc4b8e46ef187a7a => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:48.895229 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07ced8-0ffa-487e-b91f-9319231afc2d admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:48.899165 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4a07ced8-0ffa-487e-b91f-9319231afc2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:49.100714 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:49.118036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07ced8-0ffa-487e-b91f-9319231afc2d admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:49.124193 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 866/1715] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Jul 18 14:08:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:08:49.124974 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84b5e6-021b-49e1-9a53-0a80b5b90a71 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:49.125274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84b5e6-021b-49e1-9a53-0a80b5b90a71 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:49.125552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84b5e6-021b-49e1-9a53-0a80b5b90a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:49.131922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb58ba0-5823-4e74-866b-2090e35510d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:49.132924 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb58ba0-5823-4e74-866b-2090e35510d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:49.133213 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb58ba0-5823-4e74-866b-2090e35510d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:49.133490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb58ba0-5823-4e74-866b-2090e35510d8 admin admin] PATH_INFO: `/v3/projects/9d7c0a3be6034542891b016964d8d70e` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:49.136118 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb58ba0-5823-4e74-866b-2090e35510d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d7c0a3be6034542891b016964d8d70e)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:49.141325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb58ba0-5823-4e74-866b-2090e35510d8 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:49.179237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0c84b5e6-021b-49e1-9a53-0a80b5b90a71 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:49.180672 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0c84b5e6-021b-49e1-9a53-0a80b5b90a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:49.218054 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 867/1716] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:49.230926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3c44c-e451-4ade-bbac-b4918c9846f8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:49.231232 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3c44c-e451-4ade-bbac-b4918c9846f8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:49.231503 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3c44c-e451-4ade-bbac-b4918c9846f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:49.292126 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-47e3c44c-e451-4ade-bbac-b4918c9846f8 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:49.293847 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-47e3c44c-e451-4ade-bbac-b4918c9846f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:49.303952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-cbb58ba0-5823-4e74-866b-2090e35510d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d7c0a3be6034542891b016964d8d70e', 'request_id': 'req-cbb58ba0-5823-4e74-866b-2090e35510d8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:49.322442 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-cbb58ba0-5823-4e74-866b-2090e35510d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d7c0a3be6034542891b016964d8d70e', 'request_id': 'req-cbb58ba0-5823-4e74-866b-2090e35510d8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:49.329783 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:49.340549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-cbb58ba0-5823-4e74-866b-2090e35510d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d7c0a3be6034542891b016964d8d70e', 'request_id': 'req-cbb58ba0-5823-4e74-866b-2090e35510d8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:49.365921 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-cbb58ba0-5823-4e74-866b-2090e35510d8 admin admin] The token cache is being invalidate because project 9d7c0a3be6034542891b016964d8d70e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:08:49.366573 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-cbb58ba0-5823-4e74-866b-2090e35510d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:49.372639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-cbb58ba0-5823-4e74-866b-2090e35510d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:49.376845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 850/1717] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:08:48 2022] DELETE /identity/v3/projects/9d7c0a3be6034542891b016964d8d70e => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:49.381182 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb58ba0-5823-4e74-866b-2090e35510d8 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:49.384910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-cbb58ba0-5823-4e74-866b-2090e35510d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:49.429471 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 868/1718] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:49.432160 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e3c44c-e451-4ade-bbac-b4918c9846f8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:49.465734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-47e3c44c-e451-4ade-bbac-b4918c9846f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:49.543020 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb58ba0-5823-4e74-866b-2090e35510d8 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:49.558722 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9d0bd-81ec-4997-bcd3-55241013ff97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:49.559813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9d0bd-81ec-4997-bcd3-55241013ff97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:49.560105 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9d0bd-81ec-4997-bcd3-55241013ff97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:49.560549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9d0bd-81ec-4997-bcd3-55241013ff97 admin admin] PATH_INFO: `/v3/users/4bac5d3327f847d9836a86ca97b1b3cf` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:49.562401 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9d0bd-81ec-4997-bcd3-55241013ff97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bac5d3327f847d9836a86ca97b1b3cf)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:49.564521 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9d0bd-81ec-4997-bcd3-55241013ff97 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:49.661789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-35c9d0bd-81ec-4997-bcd3-55241013ff97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bac5d3327f847d9836a86ca97b1b3cf'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:49.683269 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-35c9d0bd-81ec-4997-bcd3-55241013ff97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bac5d3327f847d9836a86ca97b1b3cf'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:49.701245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-35c9d0bd-81ec-4997-bcd3-55241013ff97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bac5d3327f847d9836a86ca97b1b3cf'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:49.702698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e3c44c-e451-4ade-bbac-b4918c9846f8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:49.703595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-35c9d0bd-81ec-4997-bcd3-55241013ff97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bac5d3327f847d9836a86ca97b1b3cf'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:49.712821 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23b825-72b7-47ca-aa19-d9be37983a60 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbf513ee25374e49b3d8ce28843e393a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:49.714020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23b825-72b7-47ca-aa19-d9be37983a60 service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:49.714438 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23b825-72b7-47ca-aa19-d9be37983a60 service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:49.714645 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad23b825-72b7-47ca-aa19-d9be37983a60 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:49.723634 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-35c9d0bd-81ec-4997-bcd3-55241013ff97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bac5d3327f847d9836a86ca97b1b3cf'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:49.732726 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 851/1719] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:08:49 2022] DELETE /identity/v3/users/4bac5d3327f847d9836a86ca97b1b3cf => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:49.734423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ad23b825-72b7-47ca-aa19-d9be37983a60 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:49.734783 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9d0bd-81ec-4997-bcd3-55241013ff97 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:49.736661 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-35c9d0bd-81ec-4997-bcd3-55241013ff97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:49.845835 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9d0bd-81ec-4997-bcd3-55241013ff97 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:49.855040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42f3a5-16f3-415d-bd5f-1cfb28858ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:49.855741 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42f3a5-16f3-415d-bd5f-1cfb28858ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:49.856139 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42f3a5-16f3-415d-bd5f-1cfb28858ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:49.857035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42f3a5-16f3-415d-bd5f-1cfb28858ea0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:49.857651 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d42f3a5-16f3-415d-bd5f-1cfb28858ea0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:49.858176 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d42f3a5-16f3-415d-bd5f-1cfb28858ea0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:49.860020 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d42f3a5-16f3-415d-bd5f-1cfb28858ea0 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:49.872138 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:49.904331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 852/1720] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:49.906066 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42f3a5-16f3-415d-bd5f-1cfb28858ea0 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:49.906396 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23b825-72b7-47ca-aa19-d9be37983a60 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:49.908226 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad23b825-72b7-47ca-aa19-d9be37983a60 service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:49.922877 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42f3a5-16f3-415d-bd5f-1cfb28858ea0 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:49.931010 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379e53c-7837-4a6b-93c0-5719019d1aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:49.931250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 869/1721] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Jul 18 14:08:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:08:49.932022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379e53c-7837-4a6b-93c0-5719019d1aa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:49.932550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379e53c-7837-4a6b-93c0-5719019d1aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:49.933032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379e53c-7837-4a6b-93c0-5719019d1aa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:49.934341 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3379e53c-7837-4a6b-93c0-5719019d1aa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:49.939990 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3379e53c-7837-4a6b-93c0-5719019d1aa1 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:50.032144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 853/1722] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:08:49 2022] POST /identity/v3/projects => generated 405 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:50.037151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23b825-72b7-47ca-aa19-d9be37983a60 service swift] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:50.059018 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad23b825-72b7-47ca-aa19-d9be37983a60 service swift] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:50.121495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:50.234351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858ba77-0d44-4bd8-a7ac-357123ed4d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:50.235047 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858ba77-0d44-4bd8-a7ac-357123ed4d05 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:50.235339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858ba77-0d44-4bd8-a7ac-357123ed4d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:50.235648 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858ba77-0d44-4bd8-a7ac-357123ed4d05 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:50.236476 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858ba77-0d44-4bd8-a7ac-357123ed4d05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:50.238452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858ba77-0d44-4bd8-a7ac-357123ed4d05 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:50.317874 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379e53c-7837-4a6b-93c0-5719019d1aa1 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:50.345310 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 870/1723] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:08:50 2022] POST /identity/v3/users => generated 303 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:50.386537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379e53c-7837-4a6b-93c0-5719019d1aa1 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:50.399681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c01aa4-57f6-4879-b5b5-437ebfdb7c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:50.401912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c01aa4-57f6-4879-b5b5-437ebfdb7c8e admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:50.402206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c01aa4-57f6-4879-b5b5-437ebfdb7c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:50.402489 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c01aa4-57f6-4879-b5b5-437ebfdb7c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:50.403241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c01aa4-57f6-4879-b5b5-437ebfdb7c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:50.410113 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c01aa4-57f6-4879-b5b5-437ebfdb7c8e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:50.444457 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 854/1724] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:50.448212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858ba77-0d44-4bd8-a7ac-357123ed4d05 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:50.474979 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858ba77-0d44-4bd8-a7ac-357123ed4d05 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:50.511830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80629dfb-fd51-44c7-bc90-1808a6cb38d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:50.518988 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80629dfb-fd51-44c7-bc90-1808a6cb38d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:50.519189 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80629dfb-fd51-44c7-bc90-1808a6cb38d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:50.519420 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80629dfb-fd51-44c7-bc90-1808a6cb38d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:50.524293 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80629dfb-fd51-44c7-bc90-1808a6cb38d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:50.532116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80629dfb-fd51-44c7-bc90-1808a6cb38d3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:50.550697 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c01aa4-57f6-4879-b5b5-437ebfdb7c8e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:50.551362 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 871/1725] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:50.575136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c01aa4-57f6-4879-b5b5-437ebfdb7c8e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:50.581119 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3f64e-a945-4de6-9a11-cb89af83b13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:50.584249 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3f64e-a945-4de6-9a11-cb89af83b13a admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:50.586570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3f64e-a945-4de6-9a11-cb89af83b13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:50.588347 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3f64e-a945-4de6-9a11-cb89af83b13a admin admin] PATH_INFO: `/v3/projects/4d8864ce49af48c6b8be8e8ffb9156c0/users/e13b435ab0e74ee893d5aabe03d5843d/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:50.633651 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e3f64e-a945-4de6-9a11-cb89af83b13a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d8864ce49af48c6b8be8e8ffb9156c0, user_id=e13b435ab0e74ee893d5aabe03d5843d, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:50.643874 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e3f64e-a945-4de6-9a11-cb89af83b13a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:50.680124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 855/1726] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:08:50 2022] PUT /identity/v3/projects/4d8864ce49af48c6b8be8e8ffb9156c0/users/e13b435ab0e74ee893d5aabe03d5843d/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:50.682634 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1a950-f106-4bd9-bf97-4f0db4189b7b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:50.682937 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1a950-f106-4bd9-bf97-4f0db4189b7b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:50.683224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1a950-f106-4bd9-bf97-4f0db4189b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:50.723925 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-49a1a950-f106-4bd9-bf97-4f0db4189b7b None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:50.730291 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-49a1a950-f106-4bd9-bf97-4f0db4189b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:50.966879 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 872/1727] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:51.365304 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3f64e-a945-4de6-9a11-cb89af83b13a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:51.367252 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-35e3f64e-a945-4de6-9a11-cb89af83b13a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:51.644964 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3f64e-a945-4de6-9a11-cb89af83b13a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:51.664718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdbbc74-6191-43dd-b4eb-bf145b26e87c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:51.668250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdbbc74-6191-43dd-b4eb-bf145b26e87c service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:51.668872 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdbbc74-6191-43dd-b4eb-bf145b26e87c service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:51.669450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdbbc74-6191-43dd-b4eb-bf145b26e87c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:51.749940 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdbbc74-6191-43dd-b4eb-bf145b26e87c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:51.751425 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdbbc74-6191-43dd-b4eb-bf145b26e87c service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:51.807654 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 856/1728] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:52.111316 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef939beb-6910-4af8-9d18-5ac730c9c39a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:52.112485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef939beb-6910-4af8-9d18-5ac730c9c39a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:52.112485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef939beb-6910-4af8-9d18-5ac730c9c39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:52.156963 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ef939beb-6910-4af8-9d18-5ac730c9c39a None None] MFA Rules not processed for user `14027a0e13fd4272bd7803e3131b6348`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:52.165166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ef939beb-6910-4af8-9d18-5ac730c9c39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:52.197399 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 873/1729] 10.222.0.87 () {58 vars in 1015 bytes} [Mon Jul 18 14:08:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:52.234030 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdbbc74-6191-43dd-b4eb-bf145b26e87c service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:52.235547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6bdbbc74-6191-43dd-b4eb-bf145b26e87c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:52.264205 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdbbc74-6191-43dd-b4eb-bf145b26e87c service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:52.275582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095588c-38e6-479d-836c-e76c8f5b3c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:52.276800 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095588c-38e6-479d-836c-e76c8f5b3c7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:52.277248 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095588c-38e6-479d-836c-e76c8f5b3c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:52.277674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095588c-38e6-479d-836c-e76c8f5b3c7a admin admin] PATH_INFO: `/v3/projects/c3667a71bb064e738026bb620657d600` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:52.282520 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3095588c-38e6-479d-836c-e76c8f5b3c7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3667a71bb064e738026bb620657d600)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:52.285172 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3095588c-38e6-479d-836c-e76c8f5b3c7a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:52.436650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3095588c-38e6-479d-836c-e76c8f5b3c7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3667a71bb064e738026bb620657d600', 'request_id': 'req-3095588c-38e6-479d-836c-e76c8f5b3c7a'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:52.464800 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3095588c-38e6-479d-836c-e76c8f5b3c7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3667a71bb064e738026bb620657d600', 'request_id': 'req-3095588c-38e6-479d-836c-e76c8f5b3c7a'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:52.488465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3095588c-38e6-479d-836c-e76c8f5b3c7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3667a71bb064e738026bb620657d600', 'request_id': 'req-3095588c-38e6-479d-836c-e76c8f5b3c7a'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:52.506147 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3095588c-38e6-479d-836c-e76c8f5b3c7a admin admin] The token cache is being invalidate because project c3667a71bb064e738026bb620657d600 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:08:52.506447 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3095588c-38e6-479d-836c-e76c8f5b3c7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:52.507272 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3095588c-38e6-479d-836c-e76c8f5b3c7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:52.511884 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 857/1730] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:08:52 2022] DELETE /identity/v3/projects/c3667a71bb064e738026bb620657d600 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:52.516310 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef939beb-6910-4af8-9d18-5ac730c9c39a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:52.536632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef939beb-6910-4af8-9d18-5ac730c9c39a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:52.671281 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7e3c0-33dd-4ea8-965d-30fa355755f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:52.673074 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7e3c0-33dd-4ea8-965d-30fa355755f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:52.680187 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7e3c0-33dd-4ea8-965d-30fa355755f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:52.682854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7e3c0-33dd-4ea8-965d-30fa355755f5 admin admin] PATH_INFO: `/v3/users/d9fbda50766d4ca98dafeedbfa2c36d2` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:52.686302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a7e3c0-33dd-4ea8-965d-30fa355755f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9fbda50766d4ca98dafeedbfa2c36d2)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:52.696366 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a7e3c0-33dd-4ea8-965d-30fa355755f5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:52.733384 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095588c-38e6-479d-836c-e76c8f5b3c7a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:52.735446 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3095588c-38e6-479d-836c-e76c8f5b3c7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:52.781857 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:52.807198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:52.868606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095588c-38e6-479d-836c-e76c8f5b3c7a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:52.881368 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fc8e4-f94e-4779-a645-4474acc5d5b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:52.882239 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fc8e4-f94e-4779-a645-4474acc5d5b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:52.882677 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fc8e4-f94e-4779-a645-4474acc5d5b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:52.885153 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fc8e4-f94e-4779-a645-4474acc5d5b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:52.893544 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b8a7e3c0-33dd-4ea8-965d-30fa355755f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9fbda50766d4ca98dafeedbfa2c36d2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:52.894863 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ae6fc8e4-f94e-4779-a645-4474acc5d5b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:52.907679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b8a7e3c0-33dd-4ea8-965d-30fa355755f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9fbda50766d4ca98dafeedbfa2c36d2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:52.921597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6fc8e4-f94e-4779-a645-4474acc5d5b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:52.923240 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6fc8e4-f94e-4779-a645-4474acc5d5b9 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:52.941819 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b8a7e3c0-33dd-4ea8-965d-30fa355755f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9fbda50766d4ca98dafeedbfa2c36d2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:52.945040 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b8a7e3c0-33dd-4ea8-965d-30fa355755f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9fbda50766d4ca98dafeedbfa2c36d2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:52.946416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 858/1731] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:08:52.948557 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49e693-d4fd-4b6f-bfbb-7a00bf9565e6 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:52.949003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49e693-d4fd-4b6f-bfbb-7a00bf9565e6 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:52.949420 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49e693-d4fd-4b6f-bfbb-7a00bf9565e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:52.981863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b8a7e3c0-33dd-4ea8-965d-30fa355755f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9fbda50766d4ca98dafeedbfa2c36d2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:53.000496 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-fd49e693-d4fd-4b6f-bfbb-7a00bf9565e6 None None] MFA Rules not processed for user `0b6cb92018ea4cc5a25c5423b9bcd7a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:53.001720 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fd49e693-d4fd-4b6f-bfbb-7a00bf9565e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:53.006317 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 874/1732] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:08:52 2022] DELETE /identity/v3/users/d9fbda50766d4ca98dafeedbfa2c36d2 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:53.150890 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 859/1733] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:53.876839 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7e3c0-33dd-4ea8-965d-30fa355755f5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:53.878880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b8a7e3c0-33dd-4ea8-965d-30fa355755f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:54.198265 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7e3c0-33dd-4ea8-965d-30fa355755f5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:54.242768 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21934c5-347a-4169-b7d8-8a6d9438d96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:54.247020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21934c5-347a-4169-b7d8-8a6d9438d96e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:54.251764 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21934c5-347a-4169-b7d8-8a6d9438d96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:54.253768 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21934c5-347a-4169-b7d8-8a6d9438d96e admin admin] PATH_INFO: `/v3/projects/3ed1174b5a5848bca12ad18c4fd6447c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:54.267772 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21934c5-347a-4169-b7d8-8a6d9438d96e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ed1174b5a5848bca12ad18c4fd6447c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:54.282336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21934c5-347a-4169-b7d8-8a6d9438d96e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:54.544284 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c21934c5-347a-4169-b7d8-8a6d9438d96e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ed1174b5a5848bca12ad18c4fd6447c', 'request_id': 'req-c21934c5-347a-4169-b7d8-8a6d9438d96e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:54.598796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c21934c5-347a-4169-b7d8-8a6d9438d96e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ed1174b5a5848bca12ad18c4fd6447c', 'request_id': 'req-c21934c5-347a-4169-b7d8-8a6d9438d96e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:54.623211 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c21934c5-347a-4169-b7d8-8a6d9438d96e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ed1174b5a5848bca12ad18c4fd6447c', 'request_id': 'req-c21934c5-347a-4169-b7d8-8a6d9438d96e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:54.648081 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c21934c5-347a-4169-b7d8-8a6d9438d96e admin admin] The token cache is being invalidate because project 3ed1174b5a5848bca12ad18c4fd6447c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:08:54.648443 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c21934c5-347a-4169-b7d8-8a6d9438d96e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:54.650125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c21934c5-347a-4169-b7d8-8a6d9438d96e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:54.653987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 875/1734] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:08:53 2022] DELETE /identity/v3/projects/3ed1174b5a5848bca12ad18c4fd6447c => generated 0 bytes in 777 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:54.659160 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0370db-e880-4f09-9237-ca12a8266702 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:54.659793 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0370db-e880-4f09-9237-ca12a8266702 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:54.660122 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0370db-e880-4f09-9237-ca12a8266702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:54.725487 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-8d0370db-e880-4f09-9237-ca12a8266702 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:54.730654 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8d0370db-e880-4f09-9237-ca12a8266702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:54.927902 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 860/1735] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:54.940049 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0090b27-358e-458e-adca-45e6b49f6595 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:54.940049 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0090b27-358e-458e-adca-45e6b49f6595 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:54.940049 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0090b27-358e-458e-adca-45e6b49f6595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:54.983117 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b0090b27-358e-458e-adca-45e6b49f6595 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:54.987696 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b0090b27-358e-458e-adca-45e6b49f6595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:55.015692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 876/1736] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:08:55.020900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0370db-e880-4f09-9237-ca12a8266702 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:55.041842 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0370db-e880-4f09-9237-ca12a8266702 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:55.058826 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02ef62-5bef-424f-a856-1ca991aa7848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:55.058826 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02ef62-5bef-424f-a856-1ca991aa7848 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:55.058826 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02ef62-5bef-424f-a856-1ca991aa7848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:55.058826 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02ef62-5bef-424f-a856-1ca991aa7848 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:55.060252 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b02ef62-5bef-424f-a856-1ca991aa7848 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:55.060252 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b02ef62-5bef-424f-a856-1ca991aa7848 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:55.060252 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b02ef62-5bef-424f-a856-1ca991aa7848 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:55.075677 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 861/1737] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:55.080307 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0090b27-358e-458e-adca-45e6b49f6595 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:55.097169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0090b27-358e-458e-adca-45e6b49f6595 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:55.114872 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04413d3-158c-4cd0-9a7d-8d493cf2f099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:55.116643 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04413d3-158c-4cd0-9a7d-8d493cf2f099 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:55.118783 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04413d3-158c-4cd0-9a7d-8d493cf2f099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:55.119074 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04413d3-158c-4cd0-9a7d-8d493cf2f099 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:55.120853 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04413d3-158c-4cd0-9a7d-8d493cf2f099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:55.127907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04413d3-158c-4cd0-9a7d-8d493cf2f099 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:55.196324 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 877/1738] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:08:55 2022] POST /identity/v3/projects => generated 407 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:55.200828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02ef62-5bef-424f-a856-1ca991aa7848 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:55.216841 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02ef62-5bef-424f-a856-1ca991aa7848 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:55.580921 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b46d10-35a9-4908-b7de-403ac3e1b702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:55.581636 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b46d10-35a9-4908-b7de-403ac3e1b702 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:55.584549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b46d10-35a9-4908-b7de-403ac3e1b702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:55.584879 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b46d10-35a9-4908-b7de-403ac3e1b702 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:55.585630 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b46d10-35a9-4908-b7de-403ac3e1b702 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:55.593191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b46d10-35a9-4908-b7de-403ac3e1b702 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:55.622376 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:55.667824 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 862/1739] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:08:55 2022] POST /identity/v3/users => generated 304 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:55.668934 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04413d3-158c-4cd0-9a7d-8d493cf2f099 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:55.676981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:55.691972 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04413d3-158c-4cd0-9a7d-8d493cf2f099 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:55.723267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f7333-b159-4021-b9e1-a90b9dfe1777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:55.723980 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f7333-b159-4021-b9e1-a90b9dfe1777 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:55.734828 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f7333-b159-4021-b9e1-a90b9dfe1777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:55.735298 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f7333-b159-4021-b9e1-a90b9dfe1777 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:55.737138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f7333-b159-4021-b9e1-a90b9dfe1777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:55.744335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f7333-b159-4021-b9e1-a90b9dfe1777 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:55.748866 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00722c2b-1e2f-4d9e-89bb-addb55779c43 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:55.749459 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00722c2b-1e2f-4d9e-89bb-addb55779c43 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:55.749962 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00722c2b-1e2f-4d9e-89bb-addb55779c43 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:55.751832 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 863/1740] 10.222.0.94 () {54 vars in 921 bytes} [Mon Jul 18 14:08:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:08:55.759571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3c5ed-9028-4a21-ba78-3e9cd531429e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:55.760243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3c5ed-9028-4a21-ba78-3e9cd531429e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:55.760783 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3c5ed-9028-4a21-ba78-3e9cd531429e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:55.797448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 878/1741] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:55.800612 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-32d3c5ed-9028-4a21-ba78-3e9cd531429e None None] MFA Rules not processed for user `ac717fcd9fc1499db12cf852d45572a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:55.802010 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-32d3c5ed-9028-4a21-ba78-3e9cd531429e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:55.807710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f7333-b159-4021-b9e1-a90b9dfe1777 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:55.847120 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f7333-b159-4021-b9e1-a90b9dfe1777 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:55.865218 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86706ff-e862-4453-be7d-29ea0ccd94ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:55.866206 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86706ff-e862-4453-be7d-29ea0ccd94ff admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:55.866614 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86706ff-e862-4453-be7d-29ea0ccd94ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:55.866839 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86706ff-e862-4453-be7d-29ea0ccd94ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:55.867852 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86706ff-e862-4453-be7d-29ea0ccd94ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:55.876322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86706ff-e862-4453-be7d-29ea0ccd94ff admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:55.919144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 879/1742] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:55.919370 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86706ff-e862-4453-be7d-29ea0ccd94ff admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:55.965027 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86706ff-e862-4453-be7d-29ea0ccd94ff admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:56.003732 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed39f58-958f-432c-8bc0-426ba8ef7a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:56.009109 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed39f58-958f-432c-8bc0-426ba8ef7a48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:56.010283 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed39f58-958f-432c-8bc0-426ba8ef7a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:56.012096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed39f58-958f-432c-8bc0-426ba8ef7a48 admin admin] PATH_INFO: `/v3/projects/13901714ca504bd697a6807adbc38822/users/8ff549a5cc874432ae5f0c2cfc9bd965/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:56.053586 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 864/1743] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Jul 18 14:08:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:08:56.055106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3c5ed-9028-4a21-ba78-3e9cd531429e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:56.058612 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-32d3c5ed-9028-4a21-ba78-3e9cd531429e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:56.073930 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed39f58-958f-432c-8bc0-426ba8ef7a48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13901714ca504bd697a6807adbc38822, user_id=8ff549a5cc874432ae5f0c2cfc9bd965, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:56.085860 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed39f58-958f-432c-8bc0-426ba8ef7a48 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:56.089450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3c5ed-9028-4a21-ba78-3e9cd531429e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:56.104833 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09adc0-f54c-4f7d-a9dd-4cd45ce0d5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:56.106723 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09adc0-f54c-4f7d-a9dd-4cd45ce0d5d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:56.107134 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09adc0-f54c-4f7d-a9dd-4cd45ce0d5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:56.107531 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09adc0-f54c-4f7d-a9dd-4cd45ce0d5d7 admin admin] PATH_INFO: `/v3/users/75e118c1d0fd4c0fa95490bf6d46f926` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:56.109122 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d09adc0-f54c-4f7d-a9dd-4cd45ce0d5d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75e118c1d0fd4c0fa95490bf6d46f926)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:56.111300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d09adc0-f54c-4f7d-a9dd-4cd45ce0d5d7 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:56.161335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 880/1744] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:08:55 2022] PUT /identity/v3/projects/13901714ca504bd697a6807adbc38822/users/8ff549a5cc874432ae5f0c2cfc9bd965/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:56.162593 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed39f58-958f-432c-8bc0-426ba8ef7a48 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:56.170585 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7ed39f58-958f-432c-8bc0-426ba8ef7a48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:56.210719 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5d09adc0-f54c-4f7d-a9dd-4cd45ce0d5d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75e118c1d0fd4c0fa95490bf6d46f926'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:56.226466 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5d09adc0-f54c-4f7d-a9dd-4cd45ce0d5d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75e118c1d0fd4c0fa95490bf6d46f926'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:56.240338 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5d09adc0-f54c-4f7d-a9dd-4cd45ce0d5d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75e118c1d0fd4c0fa95490bf6d46f926'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:56.244424 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5d09adc0-f54c-4f7d-a9dd-4cd45ce0d5d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75e118c1d0fd4c0fa95490bf6d46f926'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:56.282254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5d09adc0-f54c-4f7d-a9dd-4cd45ce0d5d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75e118c1d0fd4c0fa95490bf6d46f926'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:56.290700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 865/1745] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:08:56 2022] DELETE /identity/v3/users/75e118c1d0fd4c0fa95490bf6d46f926 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:56.292070 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef949b2f-a970-430e-bd43-441e156e2741 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:56.292368 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef949b2f-a970-430e-bd43-441e156e2741 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:56.292655 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef949b2f-a970-430e-bd43-441e156e2741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:56.352373 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed39f58-958f-432c-8bc0-426ba8ef7a48 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:56.367687 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ef949b2f-a970-430e-bd43-441e156e2741 None None] MFA Rules not processed for user `8ff549a5cc874432ae5f0c2cfc9bd965`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:56.370220 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ef949b2f-a970-430e-bd43-441e156e2741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:56.505601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:56.560785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 866/1746] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:56 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:56.561730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c572a42-79aa-4a77-b96a-88820bd103d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:56.563006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f4dde-3eeb-45b7-921e-d4ae7c90aff8 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:56.563572 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f4dde-3eeb-45b7-921e-d4ae7c90aff8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:56.564351 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f4dde-3eeb-45b7-921e-d4ae7c90aff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:56.564990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c572a42-79aa-4a77-b96a-88820bd103d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:56.565289 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c572a42-79aa-4a77-b96a-88820bd103d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:56.565573 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c572a42-79aa-4a77-b96a-88820bd103d7 admin admin] PATH_INFO: `/v3/projects/c70edb2715d64ceaab313111a78ed840` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:56.570543 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c572a42-79aa-4a77-b96a-88820bd103d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c70edb2715d64ceaab313111a78ed840)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:56.576887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c572a42-79aa-4a77-b96a-88820bd103d7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:56.658202 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-648f4dde-3eeb-45b7-921e-d4ae7c90aff8 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:56.662945 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-648f4dde-3eeb-45b7-921e-d4ae7c90aff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:56.697136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 867/1747] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:56.702243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f4dde-3eeb-45b7-921e-d4ae7c90aff8 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:56.707911 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-648f4dde-3eeb-45b7-921e-d4ae7c90aff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:56.744807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3c572a42-79aa-4a77-b96a-88820bd103d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c70edb2715d64ceaab313111a78ed840', 'request_id': 'req-3c572a42-79aa-4a77-b96a-88820bd103d7'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:56.751959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f4dde-3eeb-45b7-921e-d4ae7c90aff8 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:56.767248 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3c572a42-79aa-4a77-b96a-88820bd103d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c70edb2715d64ceaab313111a78ed840', 'request_id': 'req-3c572a42-79aa-4a77-b96a-88820bd103d7'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:56.788084 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3c572a42-79aa-4a77-b96a-88820bd103d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c70edb2715d64ceaab313111a78ed840', 'request_id': 'req-3c572a42-79aa-4a77-b96a-88820bd103d7'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:56.803838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:56.828929 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3c572a42-79aa-4a77-b96a-88820bd103d7 admin admin] The token cache is being invalidate because project c70edb2715d64ceaab313111a78ed840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:08:56.829233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3c572a42-79aa-4a77-b96a-88820bd103d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:56.832324 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3c572a42-79aa-4a77-b96a-88820bd103d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:56.842129 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 881/1748] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:08:56 2022] DELETE /identity/v3/projects/c70edb2715d64ceaab313111a78ed840 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:56.843665 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bc2f7-ec37-48df-816e-772f7bb36002 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:56.843973 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bc2f7-ec37-48df-816e-772f7bb36002 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:56.844280 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bc2f7-ec37-48df-816e-772f7bb36002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:56.861566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115ed9d-c8c6-4c6f-8f1c-507009434417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:56.862601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115ed9d-c8c6-4c6f-8f1c-507009434417 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:56.862993 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115ed9d-c8c6-4c6f-8f1c-507009434417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:56.863381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115ed9d-c8c6-4c6f-8f1c-507009434417 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:56.865041 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5115ed9d-c8c6-4c6f-8f1c-507009434417 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:56.868187 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5115ed9d-c8c6-4c6f-8f1c-507009434417 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:56.910639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-847bc2f7-ec37-48df-816e-772f7bb36002 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:56.915433 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-847bc2f7-ec37-48df-816e-772f7bb36002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:56.925969 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 868/1749] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:08:56 2022] POST /identity/v3/projects => generated 407 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:56.929778 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115ed9d-c8c6-4c6f-8f1c-507009434417 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:56.941892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 882/1750] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:56.954853 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3a7f0-98ca-4486-aee5-a35fdb556305 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:56.955196 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3a7f0-98ca-4486-aee5-a35fdb556305 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:56.955448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3a7f0-98ca-4486-aee5-a35fdb556305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:56.957939 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5115ed9d-c8c6-4c6f-8f1c-507009434417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:57.032564 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-89e3a7f0-98ca-4486-aee5-a35fdb556305 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:57.034325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-89e3a7f0-98ca-4486-aee5-a35fdb556305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:57.185062 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115ed9d-c8c6-4c6f-8f1c-507009434417 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:57.195018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77258e-550f-430a-9391-973c8ded12ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:57.197285 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77258e-550f-430a-9391-973c8ded12ba admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:57.198185 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77258e-550f-430a-9391-973c8ded12ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:57.198493 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77258e-550f-430a-9391-973c8ded12ba admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:57.201474 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77258e-550f-430a-9391-973c8ded12ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:57.217454 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77258e-550f-430a-9391-973c8ded12ba admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:57.241351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 883/1751] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:57.245715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3a7f0-98ca-4486-aee5-a35fdb556305 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:57.247876 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-89e3a7f0-98ca-4486-aee5-a35fdb556305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:57.303093 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 869/1752] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:08:56 2022] POST /identity/v3/users => generated 310 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:57.304858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77258e-550f-430a-9391-973c8ded12ba admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:57.343526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77258e-550f-430a-9391-973c8ded12ba admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:57.358628 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208aa74-b8a2-4276-8b4b-a77eb0b2d87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:57.363514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208aa74-b8a2-4276-8b4b-a77eb0b2d87c admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:57.363902 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208aa74-b8a2-4276-8b4b-a77eb0b2d87c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:57.364243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208aa74-b8a2-4276-8b4b-a77eb0b2d87c admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:57.365353 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208aa74-b8a2-4276-8b4b-a77eb0b2d87c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:57.369231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208aa74-b8a2-4276-8b4b-a77eb0b2d87c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:57.371131 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208aa74-b8a2-4276-8b4b-a77eb0b2d87c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:57.397715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3a7f0-98ca-4486-aee5-a35fdb556305 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:57.408072 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 870/1753] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:57.410655 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208aa74-b8a2-4276-8b4b-a77eb0b2d87c admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:57.412069 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c665cdb-f91d-4cd3-a71a-7837ebaf347e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:57.412764 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c665cdb-f91d-4cd3-a71a-7837ebaf347e service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:57.413052 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c665cdb-f91d-4cd3-a71a-7837ebaf347e service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:57.413327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c665cdb-f91d-4cd3-a71a-7837ebaf347e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:57.419226 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7c665cdb-f91d-4cd3-a71a-7837ebaf347e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:57.435265 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208aa74-b8a2-4276-8b4b-a77eb0b2d87c admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:57.457462 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c665cdb-f91d-4cd3-a71a-7837ebaf347e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:57.458950 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c665cdb-f91d-4cd3-a71a-7837ebaf347e service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:57.465709 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e48f97-c8e0-4f7c-86e2-f955b242460c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:57.467793 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e48f97-c8e0-4f7c-86e2-f955b242460c admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:57.468263 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e48f97-c8e0-4f7c-86e2-f955b242460c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:57.468648 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e48f97-c8e0-4f7c-86e2-f955b242460c admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:57.469440 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e48f97-c8e0-4f7c-86e2-f955b242460c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:57.471206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e48f97-c8e0-4f7c-86e2-f955b242460c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:57.504508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 884/1754] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:57.505485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c665cdb-f91d-4cd3-a71a-7837ebaf347e service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:57.506073 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 871/1755] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:57.507172 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e48f97-c8e0-4f7c-86e2-f955b242460c admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:57.522308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e48f97-c8e0-4f7c-86e2-f955b242460c admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:57.541284 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c665cdb-f91d-4cd3-a71a-7837ebaf347e service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:57.546947 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63f482-e5d7-4076-bbde-aff2be5dcb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:57.548423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63f482-e5d7-4076-bbde-aff2be5dcb91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:57.548718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63f482-e5d7-4076-bbde-aff2be5dcb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:57.549007 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63f482-e5d7-4076-bbde-aff2be5dcb91 admin admin] PATH_INFO: `/v3/projects/99b98c1a3a7d4adfbab28961d5acd7a1/users/a3648822de3442d888b658c538ae33a3/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:57.564377 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fddabe-f3b4-4c4f-a30d-fe5046eeca1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:57.565062 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fddabe-f3b4-4c4f-a30d-fe5046eeca1c admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:57.565393 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fddabe-f3b4-4c4f-a30d-fe5046eeca1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:57.565763 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fddabe-f3b4-4c4f-a30d-fe5046eeca1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:57.566331 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fddabe-f3b4-4c4f-a30d-fe5046eeca1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:57.568546 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fddabe-f3b4-4c4f-a30d-fe5046eeca1c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:57.591419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63f482-e5d7-4076-bbde-aff2be5dcb91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b98c1a3a7d4adfbab28961d5acd7a1, user_id=a3648822de3442d888b658c538ae33a3, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:57.595205 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63f482-e5d7-4076-bbde-aff2be5dcb91 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:57.631760 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 872/1756] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:08:57 2022] PUT /identity/v3/projects/99b98c1a3a7d4adfbab28961d5acd7a1/users/a3648822de3442d888b658c538ae33a3/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:57.638055 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63f482-e5d7-4076-bbde-aff2be5dcb91 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:57.654334 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:57.656072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 885/1757] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:08:57 2022] POST /identity/v3/projects => generated 369 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:57.660526 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fddabe-f3b4-4c4f-a30d-fe5046eeca1c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:57.860526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63f482-e5d7-4076-bbde-aff2be5dcb91 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:57.913236 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fddabe-f3b4-4c4f-a30d-fe5046eeca1c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:57.934072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f94ba-cbe0-4305-a258-dad5ee19c311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:57.938137 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f94ba-cbe0-4305-a258-dad5ee19c311 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:57.938795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f94ba-cbe0-4305-a258-dad5ee19c311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:57.939768 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f94ba-cbe0-4305-a258-dad5ee19c311 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:57.942965 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4f94ba-cbe0-4305-a258-dad5ee19c311 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:57.945822 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4f94ba-cbe0-4305-a258-dad5ee19c311 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:58.010824 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a667ed-9c8b-441e-8fdc-8baf52ba5ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:58.011540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a667ed-9c8b-441e-8fdc-8baf52ba5ffd admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:58.012339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a667ed-9c8b-441e-8fdc-8baf52ba5ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:58.012339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a667ed-9c8b-441e-8fdc-8baf52ba5ffd admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:58.012738 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:58.014310 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a667ed-9c8b-441e-8fdc-8baf52ba5ffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:58.017344 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a667ed-9c8b-441e-8fdc-8baf52ba5ffd admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:58.024206 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 886/1758] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:08:57 2022] POST /identity/v3/users => generated 285 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:58.026731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f94ba-cbe0-4305-a258-dad5ee19c311 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:58.046506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f94ba-cbe0-4305-a258-dad5ee19c311 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:58.054193 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 873/1759] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:58.057885 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a667ed-9c8b-441e-8fdc-8baf52ba5ffd admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:58.070717 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c8354-0a2e-4f77-a828-dac216e38749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:58.072028 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c8354-0a2e-4f77-a828-dac216e38749 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:58.072346 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c8354-0a2e-4f77-a828-dac216e38749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:58.072616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c8354-0a2e-4f77-a828-dac216e38749 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:58.073366 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c8354-0a2e-4f77-a828-dac216e38749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:58.075178 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c8354-0a2e-4f77-a828-dac216e38749 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:58.100747 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a667ed-9c8b-441e-8fdc-8baf52ba5ffd admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:58.104329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 887/1760] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:58.108102 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c8354-0a2e-4f77-a828-dac216e38749 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:58.111593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56596803-5c11-4281-9430-f2234ec13c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:58.113042 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56596803-5c11-4281-9430-f2234ec13c04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:58.113489 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56596803-5c11-4281-9430-f2234ec13c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:58.113934 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56596803-5c11-4281-9430-f2234ec13c04 admin admin] PATH_INFO: `/v3/projects/99b98c1a3a7d4adfbab28961d5acd7a1/users/a3648822de3442d888b658c538ae33a3/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:58.117228 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56596803-5c11-4281-9430-f2234ec13c04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b98c1a3a7d4adfbab28961d5acd7a1, user_id=a3648822de3442d888b658c538ae33a3, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:58.125397 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56596803-5c11-4281-9430-f2234ec13c04 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:58.129446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c8354-0a2e-4f77-a828-dac216e38749 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:58.147226 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a02bb-a01e-4b87-b4d8-f454043558b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:58.147811 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a02bb-a01e-4b87-b4d8-f454043558b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:58.148224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a02bb-a01e-4b87-b4d8-f454043558b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:58.148514 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a02bb-a01e-4b87-b4d8-f454043558b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:58.149468 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a02bb-a01e-4b87-b4d8-f454043558b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:58.151363 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a02bb-a01e-4b87-b4d8-f454043558b0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:58.168584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 888/1761] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:58.172471 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a02bb-a01e-4b87-b4d8-f454043558b0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:58.175361 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 874/1762] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:08:58 2022] PUT /identity/v3/projects/99b98c1a3a7d4adfbab28961d5acd7a1/users/a3648822de3442d888b658c538ae33a3/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:58.197197 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cd693-5f44-4cb3-b2d2-cc99da0a2e38 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:58.199452 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cd693-5f44-4cb3-b2d2-cc99da0a2e38 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:58.200017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cd693-5f44-4cb3-b2d2-cc99da0a2e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:58.221115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a02bb-a01e-4b87-b4d8-f454043558b0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:58.293387 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ce2cd693-5f44-4cb3-b2d2-cc99da0a2e38 None None] MFA Rules not processed for user `a3648822de3442d888b658c538ae33a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:58.297107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ce2cd693-5f44-4cb3-b2d2-cc99da0a2e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:58.364720 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef7793-3b85-494b-8ac9-b8616c1b4516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:58.368045 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef7793-3b85-494b-8ac9-b8616c1b4516 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:58.371254 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef7793-3b85-494b-8ac9-b8616c1b4516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:58.373057 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef7793-3b85-494b-8ac9-b8616c1b4516 admin admin] PATH_INFO: `/v3/projects/3a6ed1180e3645de9bdeae8945898419/users/4740fd7484454a408b571bed5d6b5665/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:58.438910 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef7793-3b85-494b-8ac9-b8616c1b4516 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a6ed1180e3645de9bdeae8945898419, user_id=4740fd7484454a408b571bed5d6b5665, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:58.453864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef7793-3b85-494b-8ac9-b8616c1b4516 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:58.502342 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 875/1763] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:58 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:58.504911 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cd693-5f44-4cb3-b2d2-cc99da0a2e38 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:58.507764 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 889/1764] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:08:58 2022] PUT /identity/v3/projects/3a6ed1180e3645de9bdeae8945898419/users/4740fd7484454a408b571bed5d6b5665/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:58.510205 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ce2cd693-5f44-4cb3-b2d2-cc99da0a2e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:58.520761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd1a6b-5e9f-4abd-aab7-e534147eea71 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:58.521265 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd1a6b-5e9f-4abd-aab7-e534147eea71 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:58.521578 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd1a6b-5e9f-4abd-aab7-e534147eea71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:58.582549 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-44bd1a6b-5e9f-4abd-aab7-e534147eea71 None None] MFA Rules not processed for user `4740fd7484454a408b571bed5d6b5665`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:58.584636 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-44bd1a6b-5e9f-4abd-aab7-e534147eea71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:58.616137 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:58.732251 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 890/1765] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:58 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:08:58.734706 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cd693-5f44-4cb3-b2d2-cc99da0a2e38 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:58.736486 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd1a6b-5e9f-4abd-aab7-e534147eea71 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:58.869375 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:58.878084 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458f529-227c-4cb2-a7bf-eb2a1d5945ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:58.878871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458f529-227c-4cb2-a7bf-eb2a1d5945ea service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:58.879115 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458f529-227c-4cb2-a7bf-eb2a1d5945ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:58.880704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458f529-227c-4cb2-a7bf-eb2a1d5945ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:58.897961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd1a6b-5e9f-4abd-aab7-e534147eea71 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:58.911567 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64be5ba-455b-4f17-be1d-b1773c011fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:58.911567 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64be5ba-455b-4f17-be1d-b1773c011fbe admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:58.911567 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64be5ba-455b-4f17-be1d-b1773c011fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:58.911964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64be5ba-455b-4f17-be1d-b1773c011fbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:58.913684 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64be5ba-455b-4f17-be1d-b1773c011fbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:58.920490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64be5ba-455b-4f17-be1d-b1773c011fbe admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:58.992488 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 891/1766] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:08:58 2022] POST /identity/v3/projects => generated 369 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:58.994365 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64be5ba-455b-4f17-be1d-b1773c011fbe admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:58.998881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a64be5ba-455b-4f17-be1d-b1773c011fbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:59.056333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458f529-227c-4cb2-a7bf-eb2a1d5945ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:59.059976 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458f529-227c-4cb2-a7bf-eb2a1d5945ea service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:59.089303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 876/1767] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:58 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:59.107795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458f529-227c-4cb2-a7bf-eb2a1d5945ea service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:59.152244 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458f529-227c-4cb2-a7bf-eb2a1d5945ea service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:59.308152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64be5ba-455b-4f17-be1d-b1773c011fbe admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:59.338469 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a5096-594c-4322-8580-69f1ffd64759 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:59.339151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a5096-594c-4322-8580-69f1ffd64759 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:59.339442 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a5096-594c-4322-8580-69f1ffd64759 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:59.339723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a5096-594c-4322-8580-69f1ffd64759 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:59.353921 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0c5a5096-594c-4322-8580-69f1ffd64759 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:59.396986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a5096-594c-4322-8580-69f1ffd64759 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:59.399124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a5096-594c-4322-8580-69f1ffd64759 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:59.414791 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31d666-cb18-48ba-ad9f-8c5ffabbf851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:59.416705 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31d666-cb18-48ba-ad9f-8c5ffabbf851 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:59.416999 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31d666-cb18-48ba-ad9f-8c5ffabbf851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:59.417700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31d666-cb18-48ba-ad9f-8c5ffabbf851 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:59.417995 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31d666-cb18-48ba-ad9f-8c5ffabbf851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:59.422680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31d666-cb18-48ba-ad9f-8c5ffabbf851 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:59.456345 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 892/1768] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:08:59.458020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c27b2-4b54-405a-a82d-884656e41f14 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:59.458571 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c27b2-4b54-405a-a82d-884656e41f14 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:59.460834 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c27b2-4b54-405a-a82d-884656e41f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:59.495492 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 877/1769] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:08:59 2022] POST /identity/v3/users => generated 291 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:59.498671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31d666-cb18-48ba-ad9f-8c5ffabbf851 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:59.545191 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ca8c27b2-4b54-405a-a82d-884656e41f14 None None] MFA Rules not processed for user `e13b435ab0e74ee893d5aabe03d5843d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:59.549379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ca8c27b2-4b54-405a-a82d-884656e41f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:59.557280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31d666-cb18-48ba-ad9f-8c5ffabbf851 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:59.576762 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943bba3-9b0c-45ba-9aca-15674c45bdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:59.581371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943bba3-9b0c-45ba-9aca-15674c45bdec admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:59.581735 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943bba3-9b0c-45ba-9aca-15674c45bdec admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:59.582026 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943bba3-9b0c-45ba-9aca-15674c45bdec admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:59.583048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943bba3-9b0c-45ba-9aca-15674c45bdec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:59.603728 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943bba3-9b0c-45ba-9aca-15674c45bdec admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:59.648929 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 878/1770] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:08:59.650194 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943bba3-9b0c-45ba-9aca-15674c45bdec admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:59.662857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d943bba3-9b0c-45ba-9aca-15674c45bdec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:59.694344 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943bba3-9b0c-45ba-9aca-15674c45bdec admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:59.712874 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa7455-aee2-41aa-a12a-561c0491de8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:59.714099 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa7455-aee2-41aa-a12a-561c0491de8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:59.714319 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa7455-aee2-41aa-a12a-561c0491de8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:59.714610 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa7455-aee2-41aa-a12a-561c0491de8b admin admin] PATH_INFO: `/v3/projects/4a1a537bce174a47aeb560b45fd0d738` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:59.723112 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa7455-aee2-41aa-a12a-561c0491de8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a1a537bce174a47aeb560b45fd0d738)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:59.725392 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa7455-aee2-41aa-a12a-561c0491de8b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:59.727329 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:59.783731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 893/1771] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:08:59 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:08:59.783731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c27b2-4b54-405a-a82d-884656e41f14 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:59.869593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-58aa7455-aee2-41aa-a12a-561c0491de8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a1a537bce174a47aeb560b45fd0d738', 'request_id': 'req-58aa7455-aee2-41aa-a12a-561c0491de8b'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:59.887457 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-58aa7455-aee2-41aa-a12a-561c0491de8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a1a537bce174a47aeb560b45fd0d738', 'request_id': 'req-58aa7455-aee2-41aa-a12a-561c0491de8b'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:59.923895 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-58aa7455-aee2-41aa-a12a-561c0491de8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a1a537bce174a47aeb560b45fd0d738', 'request_id': 'req-58aa7455-aee2-41aa-a12a-561c0491de8b'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:59.964726 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c27b2-4b54-405a-a82d-884656e41f14 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:59.975393 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:59.995286 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-58aa7455-aee2-41aa-a12a-561c0491de8b admin admin] The token cache is being invalidate because project 4a1a537bce174a47aeb560b45fd0d738 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:08:59.995593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-58aa7455-aee2-41aa-a12a-561c0491de8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:08:59.997481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-58aa7455-aee2-41aa-a12a-561c0491de8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:00.002858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 879/1772] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:08:59 2022] DELETE /identity/v3/projects/4a1a537bce174a47aeb560b45fd0d738 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:00.011320 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa7455-aee2-41aa-a12a-561c0491de8b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:00.014421 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-58aa7455-aee2-41aa-a12a-561c0491de8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:00.144934 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30ef86-1989-4d17-93c6-52e4b0a1d0c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:00.151392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30ef86-1989-4d17-93c6-52e4b0a1d0c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:00.153111 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30ef86-1989-4d17-93c6-52e4b0a1d0c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:00.153512 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30ef86-1989-4d17-93c6-52e4b0a1d0c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:00.161988 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cd30ef86-1989-4d17-93c6-52e4b0a1d0c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:00.244315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa7455-aee2-41aa-a12a-561c0491de8b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:00.258106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f94cf7-b92c-4a34-be4c-56c8589ebf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:00.262078 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f94cf7-b92c-4a34-be4c-56c8589ebf05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:00.262078 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f94cf7-b92c-4a34-be4c-56c8589ebf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:00.263393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f94cf7-b92c-4a34-be4c-56c8589ebf05 admin admin] PATH_INFO: `/v3/projects/8c7ddf93dc2540598ecc48a72efe6782/users/a4f4c75c702043a1804aa6bd4c480282/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:00.309869 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd30ef86-1989-4d17-93c6-52e4b0a1d0c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:00.312331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f94cf7-b92c-4a34-be4c-56c8589ebf05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c7ddf93dc2540598ecc48a72efe6782, user_id=a4f4c75c702043a1804aa6bd4c480282, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:00.315376 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd30ef86-1989-4d17-93c6-52e4b0a1d0c3 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:00.319338 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f94cf7-b92c-4a34-be4c-56c8589ebf05 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:00.347610 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 894/1773] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:08:59 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:00.363671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30ef86-1989-4d17-93c6-52e4b0a1d0c3 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:00.368235 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cd30ef86-1989-4d17-93c6-52e4b0a1d0c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:00.368681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 880/1774] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:00 2022] PUT /identity/v3/projects/8c7ddf93dc2540598ecc48a72efe6782/users/a4f4c75c702043a1804aa6bd4c480282/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:00.372524 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29f758-8ebe-4ad4-b286-8b380428d934 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:00.373116 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29f758-8ebe-4ad4-b286-8b380428d934 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:00.373857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29f758-8ebe-4ad4-b286-8b380428d934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:00.413549 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30ef86-1989-4d17-93c6-52e4b0a1d0c3 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:00.439601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-bc29f758-8ebe-4ad4-b286-8b380428d934 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:00.445290 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bc29f758-8ebe-4ad4-b286-8b380428d934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:00.646476 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1be2dd-b73a-4c95-b0bc-d1844007861b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:00.651233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1be2dd-b73a-4c95-b0bc-d1844007861b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:00.651972 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1be2dd-b73a-4c95-b0bc-d1844007861b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:00.653677 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1be2dd-b73a-4c95-b0bc-d1844007861b admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:00.656547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1be2dd-b73a-4c95-b0bc-d1844007861b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:00.674628 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1be2dd-b73a-4c95-b0bc-d1844007861b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:00.696931 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 881/1775] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:00.706088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29f758-8ebe-4ad4-b286-8b380428d934 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:00.742843 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29f758-8ebe-4ad4-b286-8b380428d934 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:00.749604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 895/1776] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:00 2022] POST /identity/v3/projects => generated 403 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:00.749604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86843371-a0d6-4c66-baa7-66e1cb3b1c03 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:00.749604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86843371-a0d6-4c66-baa7-66e1cb3b1c03 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:00.749604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86843371-a0d6-4c66-baa7-66e1cb3b1c03 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:00.751770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 896/1777] 10.222.0.94 () {54 vars in 921 bytes} [Mon Jul 18 14:09:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:09:00.755366 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38651526-cb1d-4ff3-bbc7-20d3224b73ec None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:00.756969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38651526-cb1d-4ff3-bbc7-20d3224b73ec None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:00.758969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38651526-cb1d-4ff3-bbc7-20d3224b73ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:00.765077 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da5f48-a5dd-47df-98b2-9302ec3c184c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:00.765769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da5f48-a5dd-47df-98b2-9302ec3c184c admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:00.766085 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da5f48-a5dd-47df-98b2-9302ec3c184c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:00.766350 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da5f48-a5dd-47df-98b2-9302ec3c184c admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:00.767145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da5f48-a5dd-47df-98b2-9302ec3c184c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:00.768819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da5f48-a5dd-47df-98b2-9302ec3c184c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:00.794451 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 882/1778] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:00.814465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da5f48-a5dd-47df-98b2-9302ec3c184c admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:00.821101 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:00.827690 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-38651526-cb1d-4ff3-bbc7-20d3224b73ec None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:00.830674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-38651526-cb1d-4ff3-bbc7-20d3224b73ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:00.835344 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da5f48-a5dd-47df-98b2-9302ec3c184c admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:00.926979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9105258-b464-4d05-b360-214a7fe640b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:00.927757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9105258-b464-4d05-b360-214a7fe640b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:00.928081 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9105258-b464-4d05-b360-214a7fe640b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:00.928496 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9105258-b464-4d05-b360-214a7fe640b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:00.929247 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9105258-b464-4d05-b360-214a7fe640b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:00.932133 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9105258-b464-4d05-b360-214a7fe640b7 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:00.946650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:00.988013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 897/1779] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:00.992427 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38651526-cb1d-4ff3-bbc7-20d3224b73ec None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:00.997636 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 883/1780] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:00 2022] POST /identity/v3/users => generated 308 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:00.998987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025a260-e68a-430a-a2ac-bf3c590a10cd None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:00.999290 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025a260-e68a-430a-a2ac-bf3c590a10cd None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:00.999551 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025a260-e68a-430a-a2ac-bf3c590a10cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:01.007620 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38651526-cb1d-4ff3-bbc7-20d3224b73ec None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:01.015463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01e908-2af7-439a-9679-e21afa8c51dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:01.017361 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01e908-2af7-439a-9679-e21afa8c51dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:01.017361 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01e908-2af7-439a-9679-e21afa8c51dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:01.017361 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01e908-2af7-439a-9679-e21afa8c51dd admin admin] PATH_INFO: `/v3/projects/8c7ddf93dc2540598ecc48a72efe6782/users/a4f4c75c702043a1804aa6bd4c480282/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:01.019907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01e908-2af7-439a-9679-e21afa8c51dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c7ddf93dc2540598ecc48a72efe6782, user_id=a4f4c75c702043a1804aa6bd4c480282, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:01.023347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01e908-2af7-439a-9679-e21afa8c51dd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:01.040263 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 898/1781] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:00 2022] PUT /identity/v3/projects/8c7ddf93dc2540598ecc48a72efe6782/users/a4f4c75c702043a1804aa6bd4c480282/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:01.050551 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f025a260-e68a-430a-a2ac-bf3c590a10cd None None] MFA Rules not processed for user `ac717fcd9fc1499db12cf852d45572a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:01.050784 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01e908-2af7-439a-9679-e21afa8c51dd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:01.051906 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f025a260-e68a-430a-a2ac-bf3c590a10cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:01.213426 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01e908-2af7-439a-9679-e21afa8c51dd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:01.223998 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4158d-d277-475c-abc0-0ceebf86ae68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:01.229305 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4158d-d277-475c-abc0-0ceebf86ae68 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:01.229849 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4158d-d277-475c-abc0-0ceebf86ae68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:01.230271 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4158d-d277-475c-abc0-0ceebf86ae68 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:01.231548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4158d-d277-475c-abc0-0ceebf86ae68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:01.232008 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4158d-d277-475c-abc0-0ceebf86ae68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:01.233983 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4158d-d277-475c-abc0-0ceebf86ae68 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:01.267044 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 899/1782] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:09:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:01.272868 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4158d-d277-475c-abc0-0ceebf86ae68 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:01.288311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4158d-d277-475c-abc0-0ceebf86ae68 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:01.293885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a2639-861b-4ac6-8ce9-b51e697a421c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:01.294694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a2639-861b-4ac6-8ce9-b51e697a421c admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:01.295210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a2639-861b-4ac6-8ce9-b51e697a421c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:01.295686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a2639-861b-4ac6-8ce9-b51e697a421c admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:01.297020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a2639-861b-4ac6-8ce9-b51e697a421c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:01.297840 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 884/1783] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Jul 18 14:09:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:01.298881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a2639-861b-4ac6-8ce9-b51e697a421c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:01.299011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f727a-40b5-41f8-8f99-287e629c7423 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:01.299261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f727a-40b5-41f8-8f99-287e629c7423 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:01.299537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f727a-40b5-41f8-8f99-287e629c7423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:01.314760 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 900/1784] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:01.315992 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a2639-861b-4ac6-8ce9-b51e697a421c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:01.329582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a2639-861b-4ac6-8ce9-b51e697a421c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:01.335740 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced88e0b-45c5-445a-bf88-4bc8fe9a8447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:01.336684 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced88e0b-45c5-445a-bf88-4bc8fe9a8447 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:01.338035 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced88e0b-45c5-445a-bf88-4bc8fe9a8447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:01.338433 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced88e0b-45c5-445a-bf88-4bc8fe9a8447 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:01.339272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced88e0b-45c5-445a-bf88-4bc8fe9a8447 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:01.341539 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced88e0b-45c5-445a-bf88-4bc8fe9a8447 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:01.344497 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-a90f727a-40b5-41f8-8f99-287e629c7423 None None] MFA Rules not processed for user `a4f4c75c702043a1804aa6bd4c480282`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:01.346038 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a90f727a-40b5-41f8-8f99-287e629c7423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:01.391511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 901/1785] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:01 2022] POST /identity/v3/projects => generated 375 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:01.396334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced88e0b-45c5-445a-bf88-4bc8fe9a8447 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:01.574378 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 885/1786] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:01 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:01.576380 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f727a-40b5-41f8-8f99-287e629c7423 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:01.580820 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a90f727a-40b5-41f8-8f99-287e629c7423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:01.602243 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced88e0b-45c5-445a-bf88-4bc8fe9a8447 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:01.611776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620d8a7-be94-453d-9e9c-7cd43e75e0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:01.617827 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620d8a7-be94-453d-9e9c-7cd43e75e0ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:01.618285 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620d8a7-be94-453d-9e9c-7cd43e75e0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:01.618686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620d8a7-be94-453d-9e9c-7cd43e75e0ff admin admin] PATH_INFO: `/v3/projects/7937dcc79a1b4458b7bb3590e360c7af/users/a62c9c7f36764554b0b0b913cf9458e0/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:01.619978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f727a-40b5-41f8-8f99-287e629c7423 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:01.638617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:01.659488 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d620d8a7-be94-453d-9e9c-7cd43e75e0ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=7937dcc79a1b4458b7bb3590e360c7af, user_id=a62c9c7f36764554b0b0b913cf9458e0, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:01.663705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d620d8a7-be94-453d-9e9c-7cd43e75e0ff admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:01.715781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 902/1787] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:01 2022] PUT /identity/v3/projects/7937dcc79a1b4458b7bb3590e360c7af/users/a62c9c7f36764554b0b0b913cf9458e0/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:01.717033 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620d8a7-be94-453d-9e9c-7cd43e75e0ff admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:01.778276 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:01.829499 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae5044-8324-4a66-8429-61650351cadb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:01.830440 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae5044-8324-4a66-8429-61650351cadb service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:01.830872 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae5044-8324-4a66-8429-61650351cadb service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:01.831280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae5044-8324-4a66-8429-61650351cadb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:01.857603 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae5044-8324-4a66-8429-61650351cadb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:01.859895 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae5044-8324-4a66-8429-61650351cadb service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:01.879729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620d8a7-be94-453d-9e9c-7cd43e75e0ff admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:01.887293 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff778c-1c9a-46f2-8044-6738e0eaca62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:01.888371 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff778c-1c9a-46f2-8044-6738e0eaca62 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:01.889245 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff778c-1c9a-46f2-8044-6738e0eaca62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:01.889767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff778c-1c9a-46f2-8044-6738e0eaca62 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:01.890847 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff778c-1c9a-46f2-8044-6738e0eaca62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:01.894688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff778c-1c9a-46f2-8044-6738e0eaca62 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:01.899643 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 886/1788] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:01.911201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae5044-8324-4a66-8429-61650351cadb service nova] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:01.913319 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-abae5044-8324-4a66-8429-61650351cadb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:01.985037 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 903/1789] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:01 2022] POST /identity/v3/users => generated 288 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:01.985037 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff778c-1c9a-46f2-8044-6738e0eaca62 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:02.019696 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff778c-1c9a-46f2-8044-6738e0eaca62 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:02.034175 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41d583-d1e6-4aa1-b2d4-ab9b61b9d2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:02.035459 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41d583-d1e6-4aa1-b2d4-ab9b61b9d2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:02.035585 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41d583-d1e6-4aa1-b2d4-ab9b61b9d2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:02.035824 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41d583-d1e6-4aa1-b2d4-ab9b61b9d2c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:02.036624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41d583-d1e6-4aa1-b2d4-ab9b61b9d2c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:02.041145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41d583-d1e6-4aa1-b2d4-ab9b61b9d2c3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:02.072354 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 904/1790] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:02.073443 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41d583-d1e6-4aa1-b2d4-ab9b61b9d2c3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:02.089074 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41d583-d1e6-4aa1-b2d4-ab9b61b9d2c3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:02.094927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef6072-fded-4c60-b057-f8af13537b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:02.095702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef6072-fded-4c60-b057-f8af13537b6d admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:02.096116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef6072-fded-4c60-b057-f8af13537b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:02.096528 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef6072-fded-4c60-b057-f8af13537b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:02.097354 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef6072-fded-4c60-b057-f8af13537b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:02.099468 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef6072-fded-4c60-b057-f8af13537b6d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:02.105604 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae5044-8324-4a66-8429-61650351cadb service nova] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:02.138955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 905/1791] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:02.140063 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef6072-fded-4c60-b057-f8af13537b6d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:02.167037 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef6072-fded-4c60-b057-f8af13537b6d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:02.180179 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6a133-ff0c-4958-ae9a-5ba3ef7c8f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:02.181888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6a133-ff0c-4958-ae9a-5ba3ef7c8f8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:02.182489 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6a133-ff0c-4958-ae9a-5ba3ef7c8f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:02.182910 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6a133-ff0c-4958-ae9a-5ba3ef7c8f8a admin admin] PATH_INFO: `/v3/projects/7937dcc79a1b4458b7bb3590e360c7af/users/a62c9c7f36764554b0b0b913cf9458e0/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:02.189120 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6a133-ff0c-4958-ae9a-5ba3ef7c8f8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7937dcc79a1b4458b7bb3590e360c7af, user_id=a62c9c7f36764554b0b0b913cf9458e0, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:02.200208 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6a133-ff0c-4958-ae9a-5ba3ef7c8f8a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:02.203211 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e981acf-4a7e-4228-b916-71ada53047c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:02.203997 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e981acf-4a7e-4228-b916-71ada53047c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:02.204726 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e981acf-4a7e-4228-b916-71ada53047c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:02.205089 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e981acf-4a7e-4228-b916-71ada53047c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:02.239852 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 906/1792] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:02 2022] PUT /identity/v3/projects/7937dcc79a1b4458b7bb3590e360c7af/users/a62c9c7f36764554b0b0b913cf9458e0/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:02.244557 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6a133-ff0c-4958-ae9a-5ba3ef7c8f8a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:02.294979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e981acf-4a7e-4228-b916-71ada53047c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:02.298527 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e981acf-4a7e-4228-b916-71ada53047c6 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:02.305904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:02.325612 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 887/1793] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:01 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:02.388958 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6a133-ff0c-4958-ae9a-5ba3ef7c8f8a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:02.394952 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f54070-df0a-45fb-8e9f-b1ea3621076a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:02.396693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f54070-df0a-45fb-8e9f-b1ea3621076a admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:02.396693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f54070-df0a-45fb-8e9f-b1ea3621076a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:02.396693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f54070-df0a-45fb-8e9f-b1ea3621076a admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:02.397468 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f54070-df0a-45fb-8e9f-b1ea3621076a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:02.399219 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f54070-df0a-45fb-8e9f-b1ea3621076a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:02.420032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e981acf-4a7e-4228-b916-71ada53047c6 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:02.430521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 907/1794] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:02.432815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e981acf-4a7e-4228-b916-71ada53047c6 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:02.446731 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02407fc2-a788-4355-8627-842fb0a02275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:02.447166 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02407fc2-a788-4355-8627-842fb0a02275 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:02.447505 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02407fc2-a788-4355-8627-842fb0a02275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:02.448029 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02407fc2-a788-4355-8627-842fb0a02275 admin admin] PATH_INFO: `/v3/projects/9617e5278e9d4980bc5ebb70cb783f7f/users/c39ebf4683ab46a481eda5bb567ee957/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:02.467824 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02407fc2-a788-4355-8627-842fb0a02275 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9617e5278e9d4980bc5ebb70cb783f7f, user_id=c39ebf4683ab46a481eda5bb567ee957, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:02.471259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02407fc2-a788-4355-8627-842fb0a02275 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:02.509836 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 888/1795] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:02 2022] PUT /identity/v3/projects/9617e5278e9d4980bc5ebb70cb783f7f/users/c39ebf4683ab46a481eda5bb567ee957/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:02.517919 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215dffaa-9a04-4a31-8d44-5be28c10d323 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:02.518140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215dffaa-9a04-4a31-8d44-5be28c10d323 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:02.518433 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215dffaa-9a04-4a31-8d44-5be28c10d323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:02.624789 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-215dffaa-9a04-4a31-8d44-5be28c10d323 None None] MFA Rules not processed for user `c39ebf4683ab46a481eda5bb567ee957`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:02.628691 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-215dffaa-9a04-4a31-8d44-5be28c10d323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:02.804738 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:02.823669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 908/1796] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:03.404765 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02407fc2-a788-4355-8627-842fb0a02275 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:03.407143 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-02407fc2-a788-4355-8627-842fb0a02275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:03.611920 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02407fc2-a788-4355-8627-842fb0a02275 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:03.630807 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056235d-737c-4346-99d7-65f6d0897b38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:03.631619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056235d-737c-4346-99d7-65f6d0897b38 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:03.632125 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056235d-737c-4346-99d7-65f6d0897b38 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:03.632522 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056235d-737c-4346-99d7-65f6d0897b38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:03.659718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056235d-737c-4346-99d7-65f6d0897b38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:03.661397 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056235d-737c-4346-99d7-65f6d0897b38 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:03.677133 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 889/1797] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:03.719883 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1dd55-19bf-493f-9194-e08bd59c5dc7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:03.719883 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1dd55-19bf-493f-9194-e08bd59c5dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:03.719883 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1dd55-19bf-493f-9194-e08bd59c5dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:03.765064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cdb1dd55-19bf-493f-9194-e08bd59c5dc7 None None] MFA Rules not processed for user `a62c9c7f36764554b0b0b913cf9458e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:03.771475 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cdb1dd55-19bf-493f-9194-e08bd59c5dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:03.903474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 909/1798] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:03 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:04.106450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056235d-737c-4346-99d7-65f6d0897b38 service nova] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:04.197117 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056235d-737c-4346-99d7-65f6d0897b38 service nova] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:04.203862 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d99c7a-0882-42d9-82b0-1de536bc723a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:04.204975 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d99c7a-0882-42d9-82b0-1de536bc723a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:04.205305 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d99c7a-0882-42d9-82b0-1de536bc723a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:04.205683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d99c7a-0882-42d9-82b0-1de536bc723a admin admin] PATH_INFO: `/v3/users/c39ebf4683ab46a481eda5bb567ee957` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:04.207175 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d99c7a-0882-42d9-82b0-1de536bc723a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c39ebf4683ab46a481eda5bb567ee957)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:04.209706 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d99c7a-0882-42d9-82b0-1de536bc723a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:04.280542 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-97d99c7a-0882-42d9-82b0-1de536bc723a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c39ebf4683ab46a481eda5bb567ee957'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:04.289652 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-97d99c7a-0882-42d9-82b0-1de536bc723a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c39ebf4683ab46a481eda5bb567ee957'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:04.298607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-97d99c7a-0882-42d9-82b0-1de536bc723a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c39ebf4683ab46a481eda5bb567ee957'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:04.299351 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-97d99c7a-0882-42d9-82b0-1de536bc723a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c39ebf4683ab46a481eda5bb567ee957'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:04.300744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:04.318619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-97d99c7a-0882-42d9-82b0-1de536bc723a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c39ebf4683ab46a481eda5bb567ee957'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:04.326502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 890/1799] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:04 2022] DELETE /identity/v3/users/c39ebf4683ab46a481eda5bb567ee957 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:04.331743 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653c94f-004f-40c6-a3b7-b2d1abd45d64 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:04.332721 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653c94f-004f-40c6-a3b7-b2d1abd45d64 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:04.333020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653c94f-004f-40c6-a3b7-b2d1abd45d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:04.352783 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1653c94f-004f-40c6-a3b7-b2d1abd45d64 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:04.353923 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1653c94f-004f-40c6-a3b7-b2d1abd45d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:04.438340 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 910/1800] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:09:04.446100 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d99c7a-0882-42d9-82b0-1de536bc723a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:04.448906 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-97d99c7a-0882-42d9-82b0-1de536bc723a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:04.511489 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d99c7a-0882-42d9-82b0-1de536bc723a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:04.515176 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4efc4c2-a6e0-40c9-bb7e-ff21b1e50833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:04.515864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efc4c2-a6e0-40c9-bb7e-ff21b1e50833 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:04.516227 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efc4c2-a6e0-40c9-bb7e-ff21b1e50833 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:04.516514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efc4c2-a6e0-40c9-bb7e-ff21b1e50833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:04.526703 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4efc4c2-a6e0-40c9-bb7e-ff21b1e50833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:04.528194 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4efc4c2-a6e0-40c9-bb7e-ff21b1e50833 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:04.538160 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 891/1801] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:09:04.595318 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653c94f-004f-40c6-a3b7-b2d1abd45d64 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:04.597379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1653c94f-004f-40c6-a3b7-b2d1abd45d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:04.611156 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653c94f-004f-40c6-a3b7-b2d1abd45d64 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:04.619075 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daacfea4-3846-4911-a540-18b5d1bbe359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:04.620473 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacfea4-3846-4911-a540-18b5d1bbe359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:04.620894 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacfea4-3846-4911-a540-18b5d1bbe359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:04.621328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacfea4-3846-4911-a540-18b5d1bbe359 admin admin] PATH_INFO: `/v3/users/4740fd7484454a408b571bed5d6b5665` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:04.624914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daacfea4-3846-4911-a540-18b5d1bbe359 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4740fd7484454a408b571bed5d6b5665)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:04.627306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daacfea4-3846-4911-a540-18b5d1bbe359 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:04.695619 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-daacfea4-3846-4911-a540-18b5d1bbe359 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4740fd7484454a408b571bed5d6b5665'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:04.708322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-daacfea4-3846-4911-a540-18b5d1bbe359 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4740fd7484454a408b571bed5d6b5665'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:04.726842 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-daacfea4-3846-4911-a540-18b5d1bbe359 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4740fd7484454a408b571bed5d6b5665'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:04.728269 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-daacfea4-3846-4911-a540-18b5d1bbe359 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4740fd7484454a408b571bed5d6b5665'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:04.749257 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:04.750736 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-daacfea4-3846-4911-a540-18b5d1bbe359 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4740fd7484454a408b571bed5d6b5665'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:04.764000 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 911/1802] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:04 2022] DELETE /identity/v3/users/4740fd7484454a408b571bed5d6b5665 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:04.765527 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429374a6-bee9-4ec1-be62-a0a7694a35c5 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:04.765872 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429374a6-bee9-4ec1-be62-a0a7694a35c5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:04.766185 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429374a6-bee9-4ec1-be62-a0a7694a35c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:04.782079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e63b3-a79f-4faf-8a9c-8ad271675fc9 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:04.782856 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e63b3-a79f-4faf-8a9c-8ad271675fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:04.783441 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e63b3-a79f-4faf-8a9c-8ad271675fc9 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:04.785814 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 912/1803] 10.222.0.94 () {54 vars in 921 bytes} [Mon Jul 18 14:09:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:09:04.786159 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-429374a6-bee9-4ec1-be62-a0a7694a35c5 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:04.787444 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-429374a6-bee9-4ec1-be62-a0a7694a35c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:04.816716 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48951fff-eb6b-4a95-82f4-a64d34453a7a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:04.817241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48951fff-eb6b-4a95-82f4-a64d34453a7a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:04.817785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48951fff-eb6b-4a95-82f4-a64d34453a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:04.852628 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-48951fff-eb6b-4a95-82f4-a64d34453a7a None None] MFA Rules not processed for user `dd5b2ba9e37041c78a99f1700a4970c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:04.854322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-48951fff-eb6b-4a95-82f4-a64d34453a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:04.865628 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 892/1804] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:09:04.876883 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429374a6-bee9-4ec1-be62-a0a7694a35c5 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:04.880704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-429374a6-bee9-4ec1-be62-a0a7694a35c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:04.963789 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 913/1805] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Jul 18 14:09:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:04.964850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48951fff-eb6b-4a95-82f4-a64d34453a7a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:04.966213 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-48951fff-eb6b-4a95-82f4-a64d34453a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:04.981393 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48951fff-eb6b-4a95-82f4-a64d34453a7a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:04.986712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824aa26-baf4-48e4-84ab-d640e16aa671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:04.988637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824aa26-baf4-48e4-84ab-d640e16aa671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:04.988637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824aa26-baf4-48e4-84ab-d640e16aa671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:04.988919 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824aa26-baf4-48e4-84ab-d640e16aa671 admin admin] PATH_INFO: `/v3/projects/9617e5278e9d4980bc5ebb70cb783f7f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:04.990429 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2824aa26-baf4-48e4-84ab-d640e16aa671 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9617e5278e9d4980bc5ebb70cb783f7f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:04.994890 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2824aa26-baf4-48e4-84ab-d640e16aa671 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:04.996092 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429374a6-bee9-4ec1-be62-a0a7694a35c5 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:05.000188 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621e134-2863-4572-b1eb-7bad91cb7ddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:05.001129 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621e134-2863-4572-b1eb-7bad91cb7ddc service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:05.001491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621e134-2863-4572-b1eb-7bad91cb7ddc service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:05.001938 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621e134-2863-4572-b1eb-7bad91cb7ddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:05.013226 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621e134-2863-4572-b1eb-7bad91cb7ddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:05.014722 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621e134-2863-4572-b1eb-7bad91cb7ddc service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:05.031616 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 893/1806] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:05.032516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621e134-2863-4572-b1eb-7bad91cb7ddc service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:05.037775 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4621e134-2863-4572-b1eb-7bad91cb7ddc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:05.058662 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621e134-2863-4572-b1eb-7bad91cb7ddc service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:05.069395 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a673d-ea17-4957-a873-098d49d2b1a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd5b2ba9e37041c78a99f1700a4970c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:05.071267 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a673d-ea17-4957-a873-098d49d2b1a1 service placement] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:05.071791 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a673d-ea17-4957-a873-098d49d2b1a1 service placement] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:05.072279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a673d-ea17-4957-a873-098d49d2b1a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:05.087037 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a673d-ea17-4957-a873-098d49d2b1a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:05.089735 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a673d-ea17-4957-a873-098d49d2b1a1 service placement] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:05.094895 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2824aa26-baf4-48e4-84ab-d640e16aa671 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9617e5278e9d4980bc5ebb70cb783f7f', 'request_id': 'req-2824aa26-baf4-48e4-84ab-d640e16aa671'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:05.103922 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2824aa26-baf4-48e4-84ab-d640e16aa671 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9617e5278e9d4980bc5ebb70cb783f7f', 'request_id': 'req-2824aa26-baf4-48e4-84ab-d640e16aa671'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:05.106556 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 894/1807] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:05.112613 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2824aa26-baf4-48e4-84ab-d640e16aa671 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9617e5278e9d4980bc5ebb70cb783f7f', 'request_id': 'req-2824aa26-baf4-48e4-84ab-d640e16aa671'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:05.127869 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2824aa26-baf4-48e4-84ab-d640e16aa671 admin admin] The token cache is being invalidate because project 9617e5278e9d4980bc5ebb70cb783f7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:05.128495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2824aa26-baf4-48e4-84ab-d640e16aa671 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:05.129286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2824aa26-baf4-48e4-84ab-d640e16aa671 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:05.131775 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 914/1808] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:04 2022] DELETE /identity/v3/projects/9617e5278e9d4980bc5ebb70cb783f7f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:05.136772 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708aa00c-cbf5-4330-93cc-071da87f6946 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:05.137071 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708aa00c-cbf5-4330-93cc-071da87f6946 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:05.137357 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708aa00c-cbf5-4330-93cc-071da87f6946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:05.159308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-708aa00c-cbf5-4330-93cc-071da87f6946 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:05.161153 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-708aa00c-cbf5-4330-93cc-071da87f6946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:05.188591 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:05.271320 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 895/1809] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:05.280776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37100bc-4db1-45d1-8901-aa43c1dad286 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:05.281074 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37100bc-4db1-45d1-8901-aa43c1dad286 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:05.281363 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37100bc-4db1-45d1-8901-aa43c1dad286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:05.300186 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f37100bc-4db1-45d1-8901-aa43c1dad286 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:05.301403 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f37100bc-4db1-45d1-8901-aa43c1dad286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:05.310244 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 915/1810] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:05.314706 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708aa00c-cbf5-4330-93cc-071da87f6946 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:05.327668 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708aa00c-cbf5-4330-93cc-071da87f6946 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:05.333983 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc5859-0684-4fd0-a185-034329532fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:05.334669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc5859-0684-4fd0-a185-034329532fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:05.334981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc5859-0684-4fd0-a185-034329532fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:05.335246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc5859-0684-4fd0-a185-034329532fe8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:05.336004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc5859-0684-4fd0-a185-034329532fe8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:05.336320 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc5859-0684-4fd0-a185-034329532fe8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:05.338144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc5859-0684-4fd0-a185-034329532fe8 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:05.351998 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 896/1811] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:05.356796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37100bc-4db1-45d1-8901-aa43c1dad286 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:05.366284 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37100bc-4db1-45d1-8901-aa43c1dad286 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:05.372407 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bf4d4-0b81-4258-a0b5-aa558e841d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:05.373082 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bf4d4-0b81-4258-a0b5-aa558e841d12 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:05.374026 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bf4d4-0b81-4258-a0b5-aa558e841d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:05.374321 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bf4d4-0b81-4258-a0b5-aa558e841d12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:05.375040 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bf4d4-0b81-4258-a0b5-aa558e841d12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:05.377009 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bf4d4-0b81-4258-a0b5-aa558e841d12 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:05.409133 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 916/1812] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:05 2022] POST /identity/v3/projects => generated 405 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:05.413139 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc5859-0684-4fd0-a185-034329532fe8 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:05.425010 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc5859-0684-4fd0-a185-034329532fe8 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:05.475440 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bf4d4-0b81-4258-a0b5-aa558e841d12 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:05.477340 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-213bf4d4-0b81-4258-a0b5-aa558e841d12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:05.500956 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6024fd-68e3-4090-bf72-53d4fd01d746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:05.502539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6024fd-68e3-4090-bf72-53d4fd01d746 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:05.502834 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6024fd-68e3-4090-bf72-53d4fd01d746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:05.503714 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6024fd-68e3-4090-bf72-53d4fd01d746 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:05.504613 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6024fd-68e3-4090-bf72-53d4fd01d746 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:05.506573 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6024fd-68e3-4090-bf72-53d4fd01d746 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:05.535851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 897/1813] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:05 2022] POST /identity/v3/users => generated 303 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:09:05.537662 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6024fd-68e3-4090-bf72-53d4fd01d746 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:05.539266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:05.557077 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6024fd-68e3-4090-bf72-53d4fd01d746 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:05.564513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a8a6a-dbb6-40cf-9f03-f98962783c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:05.565869 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a8a6a-dbb6-40cf-9f03-f98962783c24 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:05.566472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a8a6a-dbb6-40cf-9f03-f98962783c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:05.566892 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a8a6a-dbb6-40cf-9f03-f98962783c24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:05.567837 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a8a6a-dbb6-40cf-9f03-f98962783c24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:05.568265 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bf4d4-0b81-4258-a0b5-aa558e841d12 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:05.570597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a8a6a-dbb6-40cf-9f03-f98962783c24 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:05.579358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b42796-6ef1-4d7d-8471-5fd95be59c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:05.580824 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b42796-6ef1-4d7d-8471-5fd95be59c3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:05.581529 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b42796-6ef1-4d7d-8471-5fd95be59c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:05.582393 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b42796-6ef1-4d7d-8471-5fd95be59c3f admin admin] PATH_INFO: `/v3/projects/3a6ed1180e3645de9bdeae8945898419` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:05.584873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b42796-6ef1-4d7d-8471-5fd95be59c3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a6ed1180e3645de9bdeae8945898419)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:05.588158 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b42796-6ef1-4d7d-8471-5fd95be59c3f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:05.588956 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 898/1814] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:05.593226 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a8a6a-dbb6-40cf-9f03-f98962783c24 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:05.607870 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a8a6a-dbb6-40cf-9f03-f98962783c24 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:05.614913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd82d2d7-b6fd-405a-8788-9797960964d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:05.615760 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82d2d7-b6fd-405a-8788-9797960964d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:05.616093 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82d2d7-b6fd-405a-8788-9797960964d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:05.616381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd82d2d7-b6fd-405a-8788-9797960964d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:05.617331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd82d2d7-b6fd-405a-8788-9797960964d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:05.619534 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd82d2d7-b6fd-405a-8788-9797960964d1 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:05.651797 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 899/1815] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:09:05.656720 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd82d2d7-b6fd-405a-8788-9797960964d1 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:05.676815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd82d2d7-b6fd-405a-8788-9797960964d1 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:05.687349 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-76b42796-6ef1-4d7d-8471-5fd95be59c3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a6ed1180e3645de9bdeae8945898419', 'request_id': 'req-76b42796-6ef1-4d7d-8471-5fd95be59c3f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:05.695528 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-76b42796-6ef1-4d7d-8471-5fd95be59c3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a6ed1180e3645de9bdeae8945898419', 'request_id': 'req-76b42796-6ef1-4d7d-8471-5fd95be59c3f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:05.715480 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-76b42796-6ef1-4d7d-8471-5fd95be59c3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a6ed1180e3645de9bdeae8945898419', 'request_id': 'req-76b42796-6ef1-4d7d-8471-5fd95be59c3f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:05.728500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-76b42796-6ef1-4d7d-8471-5fd95be59c3f admin admin] The token cache is being invalidate because project 3a6ed1180e3645de9bdeae8945898419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:05.729098 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-76b42796-6ef1-4d7d-8471-5fd95be59c3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:05.729686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-76b42796-6ef1-4d7d-8471-5fd95be59c3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:05.731414 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 917/1816] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:05 2022] DELETE /identity/v3/projects/3a6ed1180e3645de9bdeae8945898419 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:05.732365 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b42796-6ef1-4d7d-8471-5fd95be59c3f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:05.733635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-76b42796-6ef1-4d7d-8471-5fd95be59c3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:05.762823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e731a-a5cf-4279-b3cd-dc0cfb2beddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:05.763723 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e731a-a5cf-4279-b3cd-dc0cfb2beddb admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:05.764044 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e731a-a5cf-4279-b3cd-dc0cfb2beddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:05.764332 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e731a-a5cf-4279-b3cd-dc0cfb2beddb admin admin] PATH_INFO: `/v3/projects/6316a5bd5d3345448523bcdba2003660/users/89dce435e07c4019a6a146e9a249c7b6/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:05.776101 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:05.780008 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e731a-a5cf-4279-b3cd-dc0cfb2beddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6316a5bd5d3345448523bcdba2003660, user_id=89dce435e07c4019a6a146e9a249c7b6, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:05.783149 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e731a-a5cf-4279-b3cd-dc0cfb2beddb admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:05.797831 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 900/1817] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:05 2022] PUT /identity/v3/projects/6316a5bd5d3345448523bcdba2003660/users/89dce435e07c4019a6a146e9a249c7b6/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:05.798904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e731a-a5cf-4279-b3cd-dc0cfb2beddb admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:05.800347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b42796-6ef1-4d7d-8471-5fd95be59c3f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:05.801019 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-dc4e731a-a5cf-4279-b3cd-dc0cfb2beddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:05.867059 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e731a-a5cf-4279-b3cd-dc0cfb2beddb admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:05.871579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0a18e-00d6-42d5-a3ff-77ce78af3d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:05.872202 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51f3dc-7751-4b58-8b11-15ef72e050db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:05.872948 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0a18e-00d6-42d5-a3ff-77ce78af3d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:05.873069 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51f3dc-7751-4b58-8b11-15ef72e050db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:05.873348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51f3dc-7751-4b58-8b11-15ef72e050db admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:05.873468 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0a18e-00d6-42d5-a3ff-77ce78af3d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:05.873628 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51f3dc-7751-4b58-8b11-15ef72e050db admin admin] PATH_INFO: `/v3/users/a4f4c75c702043a1804aa6bd4c480282` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:05.873798 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0a18e-00d6-42d5-a3ff-77ce78af3d7a admin admin] PATH_INFO: `/v3/users/e13b435ab0e74ee893d5aabe03d5843d` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:05.874826 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51f3dc-7751-4b58-8b11-15ef72e050db admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4f4c75c702043a1804aa6bd4c480282)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:05.875563 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0a18e-00d6-42d5-a3ff-77ce78af3d7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e13b435ab0e74ee893d5aabe03d5843d)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:05.876832 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51f3dc-7751-4b58-8b11-15ef72e050db admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:05.878270 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0a18e-00d6-42d5-a3ff-77ce78af3d7a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:05.913313 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:05.926459 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-aa51f3dc-7751-4b58-8b11-15ef72e050db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4f4c75c702043a1804aa6bd4c480282'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:05.937967 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-aa51f3dc-7751-4b58-8b11-15ef72e050db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4f4c75c702043a1804aa6bd4c480282'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:05.944838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a0f0a18e-00d6-42d5-a3ff-77ce78af3d7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e13b435ab0e74ee893d5aabe03d5843d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:05.946607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-aa51f3dc-7751-4b58-8b11-15ef72e050db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4f4c75c702043a1804aa6bd4c480282'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:05.947376 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-aa51f3dc-7751-4b58-8b11-15ef72e050db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4f4c75c702043a1804aa6bd4c480282'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:05.959065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a0f0a18e-00d6-42d5-a3ff-77ce78af3d7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e13b435ab0e74ee893d5aabe03d5843d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:05.964971 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-aa51f3dc-7751-4b58-8b11-15ef72e050db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4f4c75c702043a1804aa6bd4c480282'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:05.967060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a0f0a18e-00d6-42d5-a3ff-77ce78af3d7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e13b435ab0e74ee893d5aabe03d5843d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:05.968661 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a0f0a18e-00d6-42d5-a3ff-77ce78af3d7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e13b435ab0e74ee893d5aabe03d5843d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:05.971978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 901/1818] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:05 2022] DELETE /identity/v3/users/a4f4c75c702043a1804aa6bd4c480282 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:05.973794 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11f760-df65-4534-9446-605179e6cc50 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:05.974294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11f760-df65-4534-9446-605179e6cc50 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:05.974583 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11f760-df65-4534-9446-605179e6cc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:05.983566 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a0f0a18e-00d6-42d5-a3ff-77ce78af3d7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e13b435ab0e74ee893d5aabe03d5843d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:05.994207 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 918/1819] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:05 2022] DELETE /identity/v3/users/e13b435ab0e74ee893d5aabe03d5843d => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:06.011723 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ac11f760-df65-4534-9446-605179e6cc50 None None] MFA Rules not processed for user `89dce435e07c4019a6a146e9a249c7b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:06.013484 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ac11f760-df65-4534-9446-605179e6cc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:06.101812 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:06.108306 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 902/1820] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:05 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:06.112361 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0a18e-00d6-42d5-a3ff-77ce78af3d7a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:06.114287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a0f0a18e-00d6-42d5-a3ff-77ce78af3d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:06.201646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0a18e-00d6-42d5-a3ff-77ce78af3d7a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:06.214380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b895b6f-51f0-4ea7-b774-4405c560e07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:06.215120 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b895b6f-51f0-4ea7-b774-4405c560e07c admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:06.215407 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b895b6f-51f0-4ea7-b774-4405c560e07c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:06.215688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b895b6f-51f0-4ea7-b774-4405c560e07c admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:06.216435 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b895b6f-51f0-4ea7-b774-4405c560e07c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:06.219660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b895b6f-51f0-4ea7-b774-4405c560e07c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:06.267822 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 919/1821] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:06 2022] POST /identity/v3/projects => generated 405 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:06.272765 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11f760-df65-4534-9446-605179e6cc50 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:06.293563 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11f760-df65-4534-9446-605179e6cc50 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:06.381753 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c6f8d-835a-4d35-a13a-1d5e47f0c6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:06.383553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c6f8d-835a-4d35-a13a-1d5e47f0c6f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:06.383553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c6f8d-835a-4d35-a13a-1d5e47f0c6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:06.383553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13c6f8d-835a-4d35-a13a-1d5e47f0c6f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:06.384445 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c6f8d-835a-4d35-a13a-1d5e47f0c6f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:06.386860 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13c6f8d-835a-4d35-a13a-1d5e47f0c6f5 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:06.410673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b895b6f-51f0-4ea7-b774-4405c560e07c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:06.412783 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6b895b6f-51f0-4ea7-b774-4405c560e07c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:06.417024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 903/1822] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:06 2022] POST /identity/v3/users => generated 309 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:06.421189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c6f8d-835a-4d35-a13a-1d5e47f0c6f5 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:06.432319 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b895b6f-51f0-4ea7-b774-4405c560e07c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:06.437528 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc836a-bf1c-47be-a290-fd2245584239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:06.438920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc836a-bf1c-47be-a290-fd2245584239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:06.438920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc836a-bf1c-47be-a290-fd2245584239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:06.439101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc836a-bf1c-47be-a290-fd2245584239 admin admin] PATH_INFO: `/v3/projects/4d8864ce49af48c6b8be8e8ffb9156c0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:06.440669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc836a-bf1c-47be-a290-fd2245584239 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d8864ce49af48c6b8be8e8ffb9156c0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:06.443974 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc836a-bf1c-47be-a290-fd2245584239 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:06.444495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13c6f8d-835a-4d35-a13a-1d5e47f0c6f5 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:06.451123 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0cb8c6-806c-42ef-8af6-7329950fce3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:06.451968 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cb8c6-806c-42ef-8af6-7329950fce3a admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:06.452267 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cb8c6-806c-42ef-8af6-7329950fce3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:06.452556 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cb8c6-806c-42ef-8af6-7329950fce3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:06.453302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0cb8c6-806c-42ef-8af6-7329950fce3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:06.459976 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0cb8c6-806c-42ef-8af6-7329950fce3a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:06.476474 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 904/1823] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:06.480254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0cb8c6-806c-42ef-8af6-7329950fce3a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:06.495061 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0cb8c6-806c-42ef-8af6-7329950fce3a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:06.502118 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479bd0e3-4014-432a-918b-621943f87e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:06.503212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479bd0e3-4014-432a-918b-621943f87e47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:06.503506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479bd0e3-4014-432a-918b-621943f87e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:06.503856 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479bd0e3-4014-432a-918b-621943f87e47 admin admin] PATH_INFO: `/v3/projects/a8d005bb1c3446c99eb484988b1fb866/users/b69ec93c8eb44c23aee179f3cfbd7f10/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:06.524272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f5fc836a-bf1c-47be-a290-fd2245584239 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d8864ce49af48c6b8be8e8ffb9156c0', 'request_id': 'req-f5fc836a-bf1c-47be-a290-fd2245584239'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:06.527813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479bd0e3-4014-432a-918b-621943f87e47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8d005bb1c3446c99eb484988b1fb866, user_id=b69ec93c8eb44c23aee179f3cfbd7f10, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:06.530508 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479bd0e3-4014-432a-918b-621943f87e47 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:06.541704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f5fc836a-bf1c-47be-a290-fd2245584239 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d8864ce49af48c6b8be8e8ffb9156c0', 'request_id': 'req-f5fc836a-bf1c-47be-a290-fd2245584239'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:06.549770 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 905/1824] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:06 2022] PUT /identity/v3/projects/a8d005bb1c3446c99eb484988b1fb866/users/b69ec93c8eb44c23aee179f3cfbd7f10/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:06.552834 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f5fc836a-bf1c-47be-a290-fd2245584239 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d8864ce49af48c6b8be8e8ffb9156c0', 'request_id': 'req-f5fc836a-bf1c-47be-a290-fd2245584239'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:06.553242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479bd0e3-4014-432a-918b-621943f87e47 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:06.555103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-479bd0e3-4014-432a-918b-621943f87e47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:06.565817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f5fc836a-bf1c-47be-a290-fd2245584239 admin admin] The token cache is being invalidate because project 4d8864ce49af48c6b8be8e8ffb9156c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:06.566143 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f5fc836a-bf1c-47be-a290-fd2245584239 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:06.567241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f5fc836a-bf1c-47be-a290-fd2245584239 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:06.569036 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 920/1825] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:06 2022] DELETE /identity/v3/projects/4d8864ce49af48c6b8be8e8ffb9156c0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:06.571876 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc836a-bf1c-47be-a290-fd2245584239 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:06.571876 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f5fc836a-bf1c-47be-a290-fd2245584239 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:06.584704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:06.626441 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479bd0e3-4014-432a-918b-621943f87e47 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:06.631191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b49b65-0033-4835-bf31-36f6e54e3956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:06.632018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b49b65-0033-4835-bf31-36f6e54e3956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:06.632316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b49b65-0033-4835-bf31-36f6e54e3956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:06.632604 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b49b65-0033-4835-bf31-36f6e54e3956 admin admin] PATH_INFO: `/v3/projects/8c7ddf93dc2540598ecc48a72efe6782` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:06.633827 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b49b65-0033-4835-bf31-36f6e54e3956 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c7ddf93dc2540598ecc48a72efe6782)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:06.635778 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b49b65-0033-4835-bf31-36f6e54e3956 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:06.639304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc836a-bf1c-47be-a290-fd2245584239 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:06.644180 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c06e8-40a8-4382-ac8d-f69753389cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:06.644880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c06e8-40a8-4382-ac8d-f69753389cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:06.645189 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c06e8-40a8-4382-ac8d-f69753389cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:06.645459 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c06e8-40a8-4382-ac8d-f69753389cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:06.646190 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c06e8-40a8-4382-ac8d-f69753389cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:06.647790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c06e8-40a8-4382-ac8d-f69753389cd4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:06.658088 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 921/1826] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:06.658880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c06e8-40a8-4382-ac8d-f69753389cd4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:06.662590 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3e6c06e8-40a8-4382-ac8d-f69753389cd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:06.677006 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c06e8-40a8-4382-ac8d-f69753389cd4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:06.694200 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d0b49b65-0033-4835-bf31-36f6e54e3956 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c7ddf93dc2540598ecc48a72efe6782', 'request_id': 'req-d0b49b65-0033-4835-bf31-36f6e54e3956'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:06.699431 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:06.701151 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d0b49b65-0033-4835-bf31-36f6e54e3956 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c7ddf93dc2540598ecc48a72efe6782', 'request_id': 'req-d0b49b65-0033-4835-bf31-36f6e54e3956'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:06.715448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d0b49b65-0033-4835-bf31-36f6e54e3956 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c7ddf93dc2540598ecc48a72efe6782', 'request_id': 'req-d0b49b65-0033-4835-bf31-36f6e54e3956'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:06.728896 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d0b49b65-0033-4835-bf31-36f6e54e3956 admin admin] The token cache is being invalidate because project 8c7ddf93dc2540598ecc48a72efe6782 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:06.729189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d0b49b65-0033-4835-bf31-36f6e54e3956 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:06.729837 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d0b49b65-0033-4835-bf31-36f6e54e3956 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:06.731414 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 906/1827] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:06 2022] DELETE /identity/v3/projects/8c7ddf93dc2540598ecc48a72efe6782 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:06.732596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b49b65-0033-4835-bf31-36f6e54e3956 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:06.734083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d0b49b65-0033-4835-bf31-36f6e54e3956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:06.742690 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b09f8-5cf2-4643-96f0-9f147eb64749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:06.744105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b09f8-5cf2-4643-96f0-9f147eb64749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:06.744396 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b09f8-5cf2-4643-96f0-9f147eb64749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:06.744697 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b09f8-5cf2-4643-96f0-9f147eb64749 admin admin] PATH_INFO: `/v3/users/a62c9c7f36764554b0b0b913cf9458e0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:06.745902 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b09f8-5cf2-4643-96f0-9f147eb64749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a62c9c7f36764554b0b0b913cf9458e0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:06.747838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b09f8-5cf2-4643-96f0-9f147eb64749 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:06.793347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-436b09f8-5cf2-4643-96f0-9f147eb64749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a62c9c7f36764554b0b0b913cf9458e0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:06.800677 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-436b09f8-5cf2-4643-96f0-9f147eb64749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a62c9c7f36764554b0b0b913cf9458e0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:06.801361 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b49b65-0033-4835-bf31-36f6e54e3956 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:06.806726 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e681ecf-bcea-4a5e-b9ca-a67b1397f3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:06.807620 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e681ecf-bcea-4a5e-b9ca-a67b1397f3a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:06.807908 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e681ecf-bcea-4a5e-b9ca-a67b1397f3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:06.808223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e681ecf-bcea-4a5e-b9ca-a67b1397f3a8 admin admin] PATH_INFO: `/v3/projects/a8d005bb1c3446c99eb484988b1fb866/users/b69ec93c8eb44c23aee179f3cfbd7f10/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:06.808792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-436b09f8-5cf2-4643-96f0-9f147eb64749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a62c9c7f36764554b0b0b913cf9458e0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:06.809458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-436b09f8-5cf2-4643-96f0-9f147eb64749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a62c9c7f36764554b0b0b913cf9458e0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:06.810185 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e681ecf-bcea-4a5e-b9ca-a67b1397f3a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8d005bb1c3446c99eb484988b1fb866, user_id=b69ec93c8eb44c23aee179f3cfbd7f10, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:06.813139 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e681ecf-bcea-4a5e-b9ca-a67b1397f3a8 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:06.822579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-436b09f8-5cf2-4643-96f0-9f147eb64749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a62c9c7f36764554b0b0b913cf9458e0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:06.824136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 907/1828] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:06 2022] PUT /identity/v3/projects/a8d005bb1c3446c99eb484988b1fb866/users/b69ec93c8eb44c23aee179f3cfbd7f10/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:06.825373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3589f-38f5-427d-9362-e195ea0727f9 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:06.825670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3589f-38f5-427d-9362-e195ea0727f9 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:06.825956 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3589f-38f5-427d-9362-e195ea0727f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:06.837311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 922/1829] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:06 2022] DELETE /identity/v3/users/a62c9c7f36764554b0b0b913cf9458e0 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:06.838687 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd3aef-12f2-447e-a169-e5446e65d078 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:06.838989 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd3aef-12f2-447e-a169-e5446e65d078 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:06.839404 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd3aef-12f2-447e-a169-e5446e65d078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:06.849092 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-04b3589f-38f5-427d-9362-e195ea0727f9 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:06.850360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-04b3589f-38f5-427d-9362-e195ea0727f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:06.877840 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-39fd3aef-12f2-447e-a169-e5446e65d078 None None] MFA Rules not processed for user `b69ec93c8eb44c23aee179f3cfbd7f10`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:06.879064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-39fd3aef-12f2-447e-a169-e5446e65d078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:06.896469 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:06.930920 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 908/1830] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:06.952560 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f3b45-c508-4de1-8aba-52658bb9bf62 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:06.952862 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f3b45-c508-4de1-8aba-52658bb9bf62 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:06.953468 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f3b45-c508-4de1-8aba-52658bb9bf62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:06.965935 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 923/1831] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:06 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:06.972624 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b36f3b45-c508-4de1-8aba-52658bb9bf62 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:06.973918 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b36f3b45-c508-4de1-8aba-52658bb9bf62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:06.976708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd3aef-12f2-447e-a169-e5446e65d078 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:06.978230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-39fd3aef-12f2-447e-a169-e5446e65d078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:06.986472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 909/1832] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:06.987901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f3b45-c508-4de1-8aba-52658bb9bf62 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:06.998511 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f3b45-c508-4de1-8aba-52658bb9bf62 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:07.003522 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f6d58-f29c-4382-9699-db1b307cfd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:07.004366 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f6d58-f29c-4382-9699-db1b307cfd0b admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:07.004727 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f6d58-f29c-4382-9699-db1b307cfd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:07.005119 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f6d58-f29c-4382-9699-db1b307cfd0b admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:07.005886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f6d58-f29c-4382-9699-db1b307cfd0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:07.006279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f6d58-f29c-4382-9699-db1b307cfd0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:07.008264 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f6d58-f29c-4382-9699-db1b307cfd0b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:07.020644 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 910/1833] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:07.025360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f6d58-f29c-4382-9699-db1b307cfd0b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:07.038579 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f6d58-f29c-4382-9699-db1b307cfd0b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:07.044662 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29179a-ea52-401e-b680-68925e990459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:07.045210 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29179a-ea52-401e-b680-68925e990459 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:07.045504 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29179a-ea52-401e-b680-68925e990459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:07.045791 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29179a-ea52-401e-b680-68925e990459 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:07.046523 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29179a-ea52-401e-b680-68925e990459 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:07.048671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29179a-ea52-401e-b680-68925e990459 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:07.054933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd3aef-12f2-447e-a169-e5446e65d078 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:07.059423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf00bc22-b608-487b-b528-b954bf49cd25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:07.060434 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00bc22-b608-487b-b528-b954bf49cd25 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:07.060729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00bc22-b608-487b-b528-b954bf49cd25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:07.061022 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00bc22-b608-487b-b528-b954bf49cd25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:07.072932 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf00bc22-b608-487b-b528-b954bf49cd25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:07.074390 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf00bc22-b608-487b-b528-b954bf49cd25 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:07.078606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 911/1834] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:07 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:07.079474 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29179a-ea52-401e-b680-68925e990459 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:07.081090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5e29179a-ea52-401e-b680-68925e990459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:07.088279 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 924/1835] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:06 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:09:07.089078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf00bc22-b608-487b-b528-b954bf49cd25 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:07.099343 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf00bc22-b608-487b-b528-b954bf49cd25 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:07.131179 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:07.167203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29179a-ea52-401e-b680-68925e990459 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:07.172354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34308572-7df1-4d15-8183-62f4fbe16697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:07.173404 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34308572-7df1-4d15-8183-62f4fbe16697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:07.173700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34308572-7df1-4d15-8183-62f4fbe16697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:07.173981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34308572-7df1-4d15-8183-62f4fbe16697 admin admin] PATH_INFO: `/v3/projects/7937dcc79a1b4458b7bb3590e360c7af` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:07.175849 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34308572-7df1-4d15-8183-62f4fbe16697 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7937dcc79a1b4458b7bb3590e360c7af)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:07.178261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34308572-7df1-4d15-8183-62f4fbe16697 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:07.190268 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c341ae-d098-4786-a1b2-531ac4877595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:07.191009 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c341ae-d098-4786-a1b2-531ac4877595 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:07.191574 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c341ae-d098-4786-a1b2-531ac4877595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:07.191840 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c341ae-d098-4786-a1b2-531ac4877595 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:07.192619 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c341ae-d098-4786-a1b2-531ac4877595 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:07.194897 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c341ae-d098-4786-a1b2-531ac4877595 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:07.230699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 925/1836] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:07 2022] POST /identity/v3/users => generated 291 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:07.234784 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c341ae-d098-4786-a1b2-531ac4877595 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:07.236493 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-34308572-7df1-4d15-8183-62f4fbe16697 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7937dcc79a1b4458b7bb3590e360c7af', 'request_id': 'req-34308572-7df1-4d15-8183-62f4fbe16697'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:07.242159 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-34308572-7df1-4d15-8183-62f4fbe16697 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7937dcc79a1b4458b7bb3590e360c7af', 'request_id': 'req-34308572-7df1-4d15-8183-62f4fbe16697'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:07.245402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c341ae-d098-4786-a1b2-531ac4877595 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:07.250201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-34308572-7df1-4d15-8183-62f4fbe16697 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7937dcc79a1b4458b7bb3590e360c7af', 'request_id': 'req-34308572-7df1-4d15-8183-62f4fbe16697'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:07.262157 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-34308572-7df1-4d15-8183-62f4fbe16697 admin admin] The token cache is being invalidate because project 7937dcc79a1b4458b7bb3590e360c7af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:07.262454 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-34308572-7df1-4d15-8183-62f4fbe16697 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:07.263101 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-34308572-7df1-4d15-8183-62f4fbe16697 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:07.265094 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 912/1837] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:07 2022] DELETE /identity/v3/projects/7937dcc79a1b4458b7bb3590e360c7af => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:07.269480 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d8735-4613-4116-b922-3cb340f3694d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:07.269773 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d8735-4613-4116-b922-3cb340f3694d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:07.270054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d8735-4613-4116-b922-3cb340f3694d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:07.288422 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b45d8735-4613-4116-b922-3cb340f3694d None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:07.291306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:07.292621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b45d8735-4613-4116-b922-3cb340f3694d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:07.329898 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e20e5a-0bc5-468f-a11b-646d9b6dca96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:07.331303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e20e5a-0bc5-468f-a11b-646d9b6dca96 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:07.331600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e20e5a-0bc5-468f-a11b-646d9b6dca96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:07.331891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e20e5a-0bc5-468f-a11b-646d9b6dca96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:07.333192 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e20e5a-0bc5-468f-a11b-646d9b6dca96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:07.338332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e20e5a-0bc5-468f-a11b-646d9b6dca96 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:07.371999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 926/1838] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:09:07.371999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e20e5a-0bc5-468f-a11b-646d9b6dca96 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:07.373076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-17e20e5a-0bc5-468f-a11b-646d9b6dca96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:07.388663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e20e5a-0bc5-468f-a11b-646d9b6dca96 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:07.393818 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6cffe3-37ce-426b-a0c9-a21fc8f4c9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:07.394772 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6cffe3-37ce-426b-a0c9-a21fc8f4c9e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:07.395143 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6cffe3-37ce-426b-a0c9-a21fc8f4c9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:07.395496 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6cffe3-37ce-426b-a0c9-a21fc8f4c9e4 admin admin] PATH_INFO: `/v3/users/89dce435e07c4019a6a146e9a249c7b6` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:07.396809 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6cffe3-37ce-426b-a0c9-a21fc8f4c9e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89dce435e07c4019a6a146e9a249c7b6)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:07.398571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 913/1839] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:07.398779 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6cffe3-37ce-426b-a0c9-a21fc8f4c9e4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:07.400592 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d8735-4613-4116-b922-3cb340f3694d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:07.405954 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b45d8735-4613-4116-b922-3cb340f3694d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:07.425285 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d8735-4613-4116-b922-3cb340f3694d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:07.432275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f9b01-04af-41a1-bf37-88472bfabf03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:07.433155 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f9b01-04af-41a1-bf37-88472bfabf03 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:07.433335 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f9b01-04af-41a1-bf37-88472bfabf03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:07.433669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f9b01-04af-41a1-bf37-88472bfabf03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:07.434494 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f9b01-04af-41a1-bf37-88472bfabf03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:07.439217 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f9b01-04af-41a1-bf37-88472bfabf03 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:07.455549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 914/1840] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:07.455549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205a5c1-0d5c-406d-8a67-4ded962eaacb None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:07.455549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205a5c1-0d5c-406d-8a67-4ded962eaacb None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:07.455549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6205a5c1-0d5c-406d-8a67-4ded962eaacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:07.462943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5d6cffe3-37ce-426b-a0c9-a21fc8f4c9e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89dce435e07c4019a6a146e9a249c7b6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:07.472095 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5d6cffe3-37ce-426b-a0c9-a21fc8f4c9e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89dce435e07c4019a6a146e9a249c7b6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:07.476575 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-6205a5c1-0d5c-406d-8a67-4ded962eaacb None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:07.478701 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6205a5c1-0d5c-406d-8a67-4ded962eaacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:07.480297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5d6cffe3-37ce-426b-a0c9-a21fc8f4c9e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89dce435e07c4019a6a146e9a249c7b6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:07.480949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5d6cffe3-37ce-426b-a0c9-a21fc8f4c9e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89dce435e07c4019a6a146e9a249c7b6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:07.490325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 915/1841] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:07.495347 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205a5c1-0d5c-406d-8a67-4ded962eaacb None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:07.496342 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5d6cffe3-37ce-426b-a0c9-a21fc8f4c9e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89dce435e07c4019a6a146e9a249c7b6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:07.497194 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6205a5c1-0d5c-406d-8a67-4ded962eaacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:07.506554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 927/1842] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:07 2022] DELETE /identity/v3/users/89dce435e07c4019a6a146e9a249c7b6 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:07.506554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6cffe3-37ce-426b-a0c9-a21fc8f4c9e4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:07.511063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6205a5c1-0d5c-406d-8a67-4ded962eaacb None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:07.585578 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:07.600743 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0d47b-c7a9-40ec-ba48-b3ffa00ede62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:07.601850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0d47b-c7a9-40ec-ba48-b3ffa00ede62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:07.602833 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0d47b-c7a9-40ec-ba48-b3ffa00ede62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:07.603284 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0d47b-c7a9-40ec-ba48-b3ffa00ede62 admin admin] PATH_INFO: `/v3/projects/d8cbb59c7e944979ae1a83cf9395e8e8/users/8f3057b35f9d4089af66fab97610aab3/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:07.624125 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0d47b-c7a9-40ec-ba48-b3ffa00ede62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8cbb59c7e944979ae1a83cf9395e8e8, user_id=8f3057b35f9d4089af66fab97610aab3, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:07.626092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6cffe3-37ce-426b-a0c9-a21fc8f4c9e4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:07.627262 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0d47b-c7a9-40ec-ba48-b3ffa00ede62 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:07.634414 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec5bab-a5d2-4e69-a114-24766737223a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:07.635307 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec5bab-a5d2-4e69-a114-24766737223a admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:07.635562 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec5bab-a5d2-4e69-a114-24766737223a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:07.635876 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec5bab-a5d2-4e69-a114-24766737223a admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:07.636774 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec5bab-a5d2-4e69-a114-24766737223a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:07.637175 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec5bab-a5d2-4e69-a114-24766737223a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:07.639160 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec5bab-a5d2-4e69-a114-24766737223a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:07.646563 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 916/1843] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:07 2022] PUT /identity/v3/projects/d8cbb59c7e944979ae1a83cf9395e8e8/users/8f3057b35f9d4089af66fab97610aab3/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:07.647796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567e9ce-f4f2-430d-9fb0-d0f0e05701ca None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:07.648244 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567e9ce-f4f2-430d-9fb0-d0f0e05701ca None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:07.648607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567e9ce-f4f2-430d-9fb0-d0f0e05701ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:07.663906 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 928/1844] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:07.665631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b2134-f055-46b4-884c-b89425f5507f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:07.666040 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b2134-f055-46b4-884c-b89425f5507f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:07.666350 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b2134-f055-46b4-884c-b89425f5507f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:07.670884 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-8567e9ce-f4f2-430d-9fb0-d0f0e05701ca None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:07.674309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8567e9ce-f4f2-430d-9fb0-d0f0e05701ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:07.694552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c32b2134-f055-46b4-884c-b89425f5507f None None] MFA Rules not processed for user `8f3057b35f9d4089af66fab97610aab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:07.695904 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c32b2134-f055-46b4-884c-b89425f5507f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:07.747984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:07.784096 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 917/1845] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:07.785485 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567e9ce-f4f2-430d-9fb0-d0f0e05701ca None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:07.807405 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567e9ce-f4f2-430d-9fb0-d0f0e05701ca None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:07.810980 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 929/1846] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:07.812060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b2134-f055-46b4-884c-b89425f5507f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:07.813798 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea067d-be0b-45bb-b629-78c93a9fa089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:07.814457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c32b2134-f055-46b4-884c-b89425f5507f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:07.814658 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea067d-be0b-45bb-b629-78c93a9fa089 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:07.815100 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea067d-be0b-45bb-b629-78c93a9fa089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:07.815550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea067d-be0b-45bb-b629-78c93a9fa089 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:07.816631 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea067d-be0b-45bb-b629-78c93a9fa089 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:07.818858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea067d-be0b-45bb-b629-78c93a9fa089 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:07.850304 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 918/1847] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:07 2022] POST /identity/v3/projects => generated 369 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:07.851137 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea067d-be0b-45bb-b629-78c93a9fa089 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:07.883368 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b2134-f055-46b4-884c-b89425f5507f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:07.948656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29696bb-6b9b-4b92-a8b8-9fd6a77145cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:07.949384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29696bb-6b9b-4b92-a8b8-9fd6a77145cd service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:07.950075 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29696bb-6b9b-4b92-a8b8-9fd6a77145cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:07.950365 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29696bb-6b9b-4b92-a8b8-9fd6a77145cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:07.962027 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29696bb-6b9b-4b92-a8b8-9fd6a77145cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:07.963542 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29696bb-6b9b-4b92-a8b8-9fd6a77145cd service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:07.969620 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea067d-be0b-45bb-b629-78c93a9fa089 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:07.975731 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba782f-5b29-4973-aaa6-24ab98e73e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:07.976450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba782f-5b29-4973-aaa6-24ab98e73e26 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:07.976741 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba782f-5b29-4973-aaa6-24ab98e73e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:07.977652 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba782f-5b29-4973-aaa6-24ab98e73e26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:07.978378 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba782f-5b29-4973-aaa6-24ab98e73e26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:07.980742 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba782f-5b29-4973-aaa6-24ab98e73e26 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:07.986386 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 930/1848] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:07.987058 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29696bb-6b9b-4b92-a8b8-9fd6a77145cd service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:07.999561 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29696bb-6b9b-4b92-a8b8-9fd6a77145cd service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:08.005818 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f6c1c-2249-42f9-90c2-96e4c209521f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:08.006636 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f6c1c-2249-42f9-90c2-96e4c209521f admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:08.006964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f6c1c-2249-42f9-90c2-96e4c209521f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:08.007315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f6c1c-2249-42f9-90c2-96e4c209521f admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:08.008198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f6c1c-2249-42f9-90c2-96e4c209521f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:08.010350 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f6c1c-2249-42f9-90c2-96e4c209521f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:08.013594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 919/1849] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:07 2022] POST /identity/v3/projects => generated 381 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:08.017614 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba782f-5b29-4973-aaa6-24ab98e73e26 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:08.076869 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 931/1850] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:07 2022] POST /identity/v3/users => generated 285 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:08.081618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f6c1c-2249-42f9-90c2-96e4c209521f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:08.095404 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:08.105623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f6c1c-2249-42f9-90c2-96e4c209521f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:08.142284 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba782f-5b29-4973-aaa6-24ab98e73e26 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:08.151163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eade6aa-d8f0-4979-9812-7d71f1bf29fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:08.152355 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eade6aa-d8f0-4979-9812-7d71f1bf29fc admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:08.152646 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eade6aa-d8f0-4979-9812-7d71f1bf29fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:08.152931 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eade6aa-d8f0-4979-9812-7d71f1bf29fc admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:08.155270 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eade6aa-d8f0-4979-9812-7d71f1bf29fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:08.157903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eade6aa-d8f0-4979-9812-7d71f1bf29fc admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:08.209961 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 920/1851] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:08 2022] POST /identity/v3/users => generated 297 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:08.236302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eade6aa-d8f0-4979-9812-7d71f1bf29fc admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:08.244865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e564f94-04b3-45ec-94b2-c0ffcd991225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:08.246032 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e564f94-04b3-45ec-94b2-c0ffcd991225 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:08.246667 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e564f94-04b3-45ec-94b2-c0ffcd991225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:08.247144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e564f94-04b3-45ec-94b2-c0ffcd991225 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:08.249121 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e564f94-04b3-45ec-94b2-c0ffcd991225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:08.251169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e564f94-04b3-45ec-94b2-c0ffcd991225 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:08.265298 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 932/1852] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:08.271057 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e564f94-04b3-45ec-94b2-c0ffcd991225 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:08.272453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eade6aa-d8f0-4979-9812-7d71f1bf29fc admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:08.285096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e564f94-04b3-45ec-94b2-c0ffcd991225 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:08.287451 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57d311-3ff8-4a9a-b8ee-e94fb0efab04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:08.288225 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57d311-3ff8-4a9a-b8ee-e94fb0efab04 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:08.288468 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57d311-3ff8-4a9a-b8ee-e94fb0efab04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:08.288746 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57d311-3ff8-4a9a-b8ee-e94fb0efab04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:08.289519 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57d311-3ff8-4a9a-b8ee-e94fb0efab04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:08.290355 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b5d4b-05ab-4b8b-a4eb-336f43cfd654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:08.291409 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b5d4b-05ab-4b8b-a4eb-336f43cfd654 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:08.291519 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57d311-3ff8-4a9a-b8ee-e94fb0efab04 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:08.291605 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b5d4b-05ab-4b8b-a4eb-336f43cfd654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:08.291605 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b5d4b-05ab-4b8b-a4eb-336f43cfd654 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:08.292356 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b5d4b-05ab-4b8b-a4eb-336f43cfd654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:08.294040 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b5d4b-05ab-4b8b-a4eb-336f43cfd654 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:08.310446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 933/1853] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:09:08.315265 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b5d4b-05ab-4b8b-a4eb-336f43cfd654 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:08.323147 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 921/1854] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:09:08.327473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57d311-3ff8-4a9a-b8ee-e94fb0efab04 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:08.330792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:08.339598 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57d311-3ff8-4a9a-b8ee-e94fb0efab04 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:08.345545 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d223e-f613-444f-be48-4be1d0f82456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:08.346543 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d223e-f613-444f-be48-4be1d0f82456 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:08.346761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b5d4b-05ab-4b8b-a4eb-336f43cfd654 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:08.346874 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d223e-f613-444f-be48-4be1d0f82456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:08.347093 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d223e-f613-444f-be48-4be1d0f82456 admin admin] PATH_INFO: `/v3/projects/ac55065a0faa4689893dfceaa7807562/users/dd0109feacf04f16838c357395098beb/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:08.353900 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce293a-f52b-46c8-8deb-af38d6e09fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:08.356319 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce293a-f52b-46c8-8deb-af38d6e09fd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:08.356613 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce293a-f52b-46c8-8deb-af38d6e09fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:08.356958 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce293a-f52b-46c8-8deb-af38d6e09fd6 admin admin] PATH_INFO: `/v3/projects/0b142446207e413ab254134f2aa4c48d/users/c5b42202b6dd4af084750ea3a025ae94/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:08.370441 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d223e-f613-444f-be48-4be1d0f82456 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac55065a0faa4689893dfceaa7807562, user_id=dd0109feacf04f16838c357395098beb, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:08.373511 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d223e-f613-444f-be48-4be1d0f82456 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:08.375558 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce293a-f52b-46c8-8deb-af38d6e09fd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b142446207e413ab254134f2aa4c48d, user_id=c5b42202b6dd4af084750ea3a025ae94, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:08.380558 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce293a-f52b-46c8-8deb-af38d6e09fd6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:08.396020 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 922/1855] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:08 2022] PUT /identity/v3/projects/ac55065a0faa4689893dfceaa7807562/users/dd0109feacf04f16838c357395098beb/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:08.397129 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d223e-f613-444f-be48-4be1d0f82456 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:08.413206 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 934/1856] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:08 2022] PUT /identity/v3/projects/0b142446207e413ab254134f2aa4c48d/users/c5b42202b6dd4af084750ea3a025ae94/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:08.414968 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57a1b7-6d02-4a0f-aa09-d08436e93ae0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:08.415233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57a1b7-6d02-4a0f-aa09-d08436e93ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:08.415485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57a1b7-6d02-4a0f-aa09-d08436e93ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:08.485542 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1c57a1b7-6d02-4a0f-aa09-d08436e93ae0 None None] MFA Rules not processed for user `c5b42202b6dd4af084750ea3a025ae94`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:08.485542 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1c57a1b7-6d02-4a0f-aa09-d08436e93ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:08.490349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d223e-f613-444f-be48-4be1d0f82456 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:08.698236 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 935/1857] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:08 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:08.701463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57a1b7-6d02-4a0f-aa09-d08436e93ae0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:08.718668 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41db3da-7596-4a1b-be89-3fc9c8927e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:08.719410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41db3da-7596-4a1b-be89-3fc9c8927e02 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:08.719835 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41db3da-7596-4a1b-be89-3fc9c8927e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:08.720296 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41db3da-7596-4a1b-be89-3fc9c8927e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:08.721243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41db3da-7596-4a1b-be89-3fc9c8927e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:08.723298 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41db3da-7596-4a1b-be89-3fc9c8927e02 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:08.726828 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57a1b7-6d02-4a0f-aa09-d08436e93ae0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:08.734127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d397889-e5a6-40f5-bb63-374ce066a950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:08.734498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d397889-e5a6-40f5-bb63-374ce066a950 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:08.734791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d397889-e5a6-40f5-bb63-374ce066a950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:08.735076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d397889-e5a6-40f5-bb63-374ce066a950 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:08.735815 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d397889-e5a6-40f5-bb63-374ce066a950 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:08.738146 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d397889-e5a6-40f5-bb63-374ce066a950 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:08.746371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 923/1858] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:08.746371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41db3da-7596-4a1b-be89-3fc9c8927e02 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:08.748325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b41db3da-7596-4a1b-be89-3fc9c8927e02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:08.767115 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41db3da-7596-4a1b-be89-3fc9c8927e02 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:08.775677 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74ff93-ab87-4c18-9b26-8b9688154702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:08.776886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74ff93-ab87-4c18-9b26-8b9688154702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:08.777338 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74ff93-ab87-4c18-9b26-8b9688154702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:08.777745 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74ff93-ab87-4c18-9b26-8b9688154702 admin admin] PATH_INFO: `/v3/projects/6316a5bd5d3345448523bcdba2003660` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:08.779203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74ff93-ab87-4c18-9b26-8b9688154702 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6316a5bd5d3345448523bcdba2003660)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:08.782612 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74ff93-ab87-4c18-9b26-8b9688154702 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:08.785523 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 936/1859] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:08 2022] POST /identity/v3/projects => generated 365 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:08.786710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d397889-e5a6-40f5-bb63-374ce066a950 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:08.842014 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:08.860811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ff74ff93-ab87-4c18-9b26-8b9688154702 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6316a5bd5d3345448523bcdba2003660', 'request_id': 'req-ff74ff93-ab87-4c18-9b26-8b9688154702'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:08.865430 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d397889-e5a6-40f5-bb63-374ce066a950 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:08.868612 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ff74ff93-ab87-4c18-9b26-8b9688154702 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6316a5bd5d3345448523bcdba2003660', 'request_id': 'req-ff74ff93-ab87-4c18-9b26-8b9688154702'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:08.878970 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ff74ff93-ab87-4c18-9b26-8b9688154702 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6316a5bd5d3345448523bcdba2003660', 'request_id': 'req-ff74ff93-ab87-4c18-9b26-8b9688154702'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:08.891632 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ff74ff93-ab87-4c18-9b26-8b9688154702 admin admin] The token cache is being invalidate because project 6316a5bd5d3345448523bcdba2003660 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:08.892208 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ff74ff93-ab87-4c18-9b26-8b9688154702 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:08.893011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ff74ff93-ab87-4c18-9b26-8b9688154702 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:08.897628 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 924/1860] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:08 2022] DELETE /identity/v3/projects/6316a5bd5d3345448523bcdba2003660 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:08.897628 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74ff93-ab87-4c18-9b26-8b9688154702 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:08.899448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ff74ff93-ab87-4c18-9b26-8b9688154702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:08.928225 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:08.954089 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0d49c-0b1a-45ef-a37e-e8d9eed9e9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:08.955886 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0d49c-0b1a-45ef-a37e-e8d9eed9e9f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:08.956495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0d49c-0b1a-45ef-a37e-e8d9eed9e9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:08.956876 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0d49c-0b1a-45ef-a37e-e8d9eed9e9f7 admin admin] PATH_INFO: `/v3/projects/ac55065a0faa4689893dfceaa7807562/users/dd0109feacf04f16838c357395098beb/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:08.959193 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0d49c-0b1a-45ef-a37e-e8d9eed9e9f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac55065a0faa4689893dfceaa7807562, user_id=dd0109feacf04f16838c357395098beb, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:08.963028 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0d49c-0b1a-45ef-a37e-e8d9eed9e9f7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:08.980117 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 937/1861] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:08 2022] PUT /identity/v3/projects/ac55065a0faa4689893dfceaa7807562/users/dd0109feacf04f16838c357395098beb/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:08.980299 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74ff93-ab87-4c18-9b26-8b9688154702 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:08.980383 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0d49c-0b1a-45ef-a37e-e8d9eed9e9f7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:08.984723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-81d0d49c-0b1a-45ef-a37e-e8d9eed9e9f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:09.047658 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d862f-23a7-4506-989c-2b1253b0a276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:09.048480 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d862f-23a7-4506-989c-2b1253b0a276 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:09.048794 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d862f-23a7-4506-989c-2b1253b0a276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:09.049076 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d862f-23a7-4506-989c-2b1253b0a276 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:09.049921 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d862f-23a7-4506-989c-2b1253b0a276 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:09.052148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d862f-23a7-4506-989c-2b1253b0a276 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:09.076317 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 925/1862] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:08 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:09.078687 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e5b9b-0f49-4611-bf86-45ad1f29d06e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:09.079518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e5b9b-0f49-4611-bf86-45ad1f29d06e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:09.079829 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e5b9b-0f49-4611-bf86-45ad1f29d06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:09.082699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0d49c-0b1a-45ef-a37e-e8d9eed9e9f7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:09.088999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1470828-037f-4b1b-855f-310cc48ff9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:09.090271 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1470828-037f-4b1b-855f-310cc48ff9d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:09.090695 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1470828-037f-4b1b-855f-310cc48ff9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:09.091288 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1470828-037f-4b1b-855f-310cc48ff9d5 admin admin] PATH_INFO: `/v3/users/b69ec93c8eb44c23aee179f3cfbd7f10` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:09.092668 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1470828-037f-4b1b-855f-310cc48ff9d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b69ec93c8eb44c23aee179f3cfbd7f10)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:09.095101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1470828-037f-4b1b-855f-310cc48ff9d5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:09.109878 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ff7e5b9b-0f49-4611-bf86-45ad1f29d06e None None] MFA Rules not processed for user `dd0109feacf04f16838c357395098beb`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:09.111047 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ff7e5b9b-0f49-4611-bf86-45ad1f29d06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:09.168449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a1470828-037f-4b1b-855f-310cc48ff9d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b69ec93c8eb44c23aee179f3cfbd7f10'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:09.176948 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:09.179450 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a1470828-037f-4b1b-855f-310cc48ff9d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b69ec93c8eb44c23aee179f3cfbd7f10'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:09.196600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a1470828-037f-4b1b-855f-310cc48ff9d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b69ec93c8eb44c23aee179f3cfbd7f10'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:09.197382 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a1470828-037f-4b1b-855f-310cc48ff9d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b69ec93c8eb44c23aee179f3cfbd7f10'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:09.227636 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a1470828-037f-4b1b-855f-310cc48ff9d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b69ec93c8eb44c23aee179f3cfbd7f10'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:09.231745 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 926/1863] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:09.240843 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 938/1864] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:08 2022] DELETE /identity/v3/users/b69ec93c8eb44c23aee179f3cfbd7f10 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:09.244282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e5b9b-0f49-4611-bf86-45ad1f29d06e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:09.245851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ff7e5b9b-0f49-4611-bf86-45ad1f29d06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:09.249090 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1470828-037f-4b1b-855f-310cc48ff9d5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:09.251506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a1470828-037f-4b1b-855f-310cc48ff9d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:09.260057 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e5b9b-0f49-4611-bf86-45ad1f29d06e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:09.326808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b5092-16a1-4d0c-8e60-f4264e4a767f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:09.327524 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b5092-16a1-4d0c-8e60-f4264e4a767f admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:09.328340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b5092-16a1-4d0c-8e60-f4264e4a767f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:09.328676 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b5092-16a1-4d0c-8e60-f4264e4a767f admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:09.329451 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b5092-16a1-4d0c-8e60-f4264e4a767f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:09.331820 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b5092-16a1-4d0c-8e60-f4264e4a767f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:09.348523 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 927/1865] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:09.349650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b5092-16a1-4d0c-8e60-f4264e4a767f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:09.363036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b5092-16a1-4d0c-8e60-f4264e4a767f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:09.369241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508782f-d868-4c8b-998d-d4312131ef36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:09.370249 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508782f-d868-4c8b-998d-d4312131ef36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:09.371231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508782f-d868-4c8b-998d-d4312131ef36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:09.371604 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508782f-d868-4c8b-998d-d4312131ef36 admin admin] PATH_INFO: `/v3/projects/2894e0d6363f42a085257394cb6dd0bc/users/941a58530932444b8cc9815783f1f8db/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:09.390421 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1470828-037f-4b1b-855f-310cc48ff9d5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:09.397376 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f556442-36e7-4660-9245-5a67cd9a9171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:09.398143 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f556442-36e7-4660-9245-5a67cd9a9171 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:09.399646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f556442-36e7-4660-9245-5a67cd9a9171 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:09.399901 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f556442-36e7-4660-9245-5a67cd9a9171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:09.402549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d508782f-d868-4c8b-998d-d4312131ef36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2894e0d6363f42a085257394cb6dd0bc, user_id=941a58530932444b8cc9815783f1f8db, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:09.405263 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d508782f-d868-4c8b-998d-d4312131ef36 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:09.426106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 928/1866] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:09 2022] PUT /identity/v3/projects/2894e0d6363f42a085257394cb6dd0bc/users/941a58530932444b8cc9815783f1f8db/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:09.427173 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508782f-d868-4c8b-998d-d4312131ef36 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:09.433107 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:09.501110 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f556442-36e7-4660-9245-5a67cd9a9171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:09.501110 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f556442-36e7-4660-9245-5a67cd9a9171 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:09.505322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 939/1867] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:09.512430 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508782f-d868-4c8b-998d-d4312131ef36 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:09.517791 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9faa903-5db6-49d9-a13f-9925b568b6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:09.518584 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9faa903-5db6-49d9-a13f-9925b568b6eb admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:09.519173 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9faa903-5db6-49d9-a13f-9925b568b6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:09.519284 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9faa903-5db6-49d9-a13f-9925b568b6eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:09.520214 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9faa903-5db6-49d9-a13f-9925b568b6eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:09.521883 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9faa903-5db6-49d9-a13f-9925b568b6eb admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:09.533937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 929/1868] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:09.538917 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f556442-36e7-4660-9245-5a67cd9a9171 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:09.553139 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f556442-36e7-4660-9245-5a67cd9a9171 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:09.559042 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a018c2-1190-42cf-97a0-0dc3065850bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:09.560125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a018c2-1190-42cf-97a0-0dc3065850bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:09.560503 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a018c2-1190-42cf-97a0-0dc3065850bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:09.561009 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a018c2-1190-42cf-97a0-0dc3065850bd admin admin] PATH_INFO: `/v3/projects/2894e0d6363f42a085257394cb6dd0bc/users/941a58530932444b8cc9815783f1f8db/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:09.563141 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a018c2-1190-42cf-97a0-0dc3065850bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2894e0d6363f42a085257394cb6dd0bc, user_id=941a58530932444b8cc9815783f1f8db, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:09.565921 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a018c2-1190-42cf-97a0-0dc3065850bd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:09.578682 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 940/1869] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:09 2022] PUT /identity/v3/projects/2894e0d6363f42a085257394cb6dd0bc/users/941a58530932444b8cc9815783f1f8db/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:09.589776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c33ab-5941-415d-889a-ff3b1ce54fc1 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:09.590274 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c33ab-5941-415d-889a-ff3b1ce54fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:09.590686 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c33ab-5941-415d-889a-ff3b1ce54fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:09.654492 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-5e3c33ab-5941-415d-889a-ff3b1ce54fc1 None None] MFA Rules not processed for user `941a58530932444b8cc9815783f1f8db`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:09.656804 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5e3c33ab-5941-415d-889a-ff3b1ce54fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:09.733834 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:09.799001 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 930/1870] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:09 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:09.805951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a018c2-1190-42cf-97a0-0dc3065850bd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:09.808467 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c1a018c2-1190-42cf-97a0-0dc3065850bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:09.834246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c33ab-5941-415d-889a-ff3b1ce54fc1 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:09.836480 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5e3c33ab-5941-415d-889a-ff3b1ce54fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:09.906485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a018c2-1190-42cf-97a0-0dc3065850bd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:09.912943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0048ff-166c-4742-a5b9-084d792d0e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:09.914085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0048ff-166c-4742-a5b9-084d792d0e5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:09.914494 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0048ff-166c-4742-a5b9-084d792d0e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:09.914842 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0048ff-166c-4742-a5b9-084d792d0e5f admin admin] PATH_INFO: `/v3/projects/a8d005bb1c3446c99eb484988b1fb866` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:09.916157 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0048ff-166c-4742-a5b9-084d792d0e5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8d005bb1c3446c99eb484988b1fb866)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:09.918739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0048ff-166c-4742-a5b9-084d792d0e5f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:09.931623 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c33ab-5941-415d-889a-ff3b1ce54fc1 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:09.935711 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a9afd-de57-4959-be6e-920044a7aa53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:09.936417 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a9afd-de57-4959-be6e-920044a7aa53 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:09.936709 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a9afd-de57-4959-be6e-920044a7aa53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:09.937118 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a9afd-de57-4959-be6e-920044a7aa53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:09.951480 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a9afd-de57-4959-be6e-920044a7aa53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:09.953089 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a9afd-de57-4959-be6e-920044a7aa53 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:09.966288 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 931/1871] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:09 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:09:09.967267 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a9afd-de57-4959-be6e-920044a7aa53 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:10.009818 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1c0048ff-166c-4742-a5b9-084d792d0e5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8d005bb1c3446c99eb484988b1fb866', 'request_id': 'req-1c0048ff-166c-4742-a5b9-084d792d0e5f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:10.021537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1c0048ff-166c-4742-a5b9-084d792d0e5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8d005bb1c3446c99eb484988b1fb866', 'request_id': 'req-1c0048ff-166c-4742-a5b9-084d792d0e5f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:10.032377 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1c0048ff-166c-4742-a5b9-084d792d0e5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8d005bb1c3446c99eb484988b1fb866', 'request_id': 'req-1c0048ff-166c-4742-a5b9-084d792d0e5f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:10.049466 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1c0048ff-166c-4742-a5b9-084d792d0e5f admin admin] The token cache is being invalidate because project a8d005bb1c3446c99eb484988b1fb866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:10.049800 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1c0048ff-166c-4742-a5b9-084d792d0e5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:10.050634 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1c0048ff-166c-4742-a5b9-084d792d0e5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:10.056311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 941/1872] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:09 2022] DELETE /identity/v3/projects/a8d005bb1c3446c99eb484988b1fb866 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:10.056311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0048ff-166c-4742-a5b9-084d792d0e5f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:10.056311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1c0048ff-166c-4742-a5b9-084d792d0e5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:10.061542 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a9afd-de57-4959-be6e-920044a7aa53 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:10.090998 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:10.115394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0048ff-166c-4742-a5b9-084d792d0e5f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:10.119555 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7867e-e620-4b68-a88d-df7baa2541a2 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd0109feacf04f16838c357395098beb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac55065a0faa4689893dfceaa7807562', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:10.119779 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bb817-8153-4ff4-bda6-424bfd2f57eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:10.120292 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7867e-e620-4b68-a88d-df7baa2541a2 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:10.120464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bb817-8153-4ff4-bda6-424bfd2f57eb service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:10.120698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7867e-e620-4b68-a88d-df7baa2541a2 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:10.120790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bb817-8153-4ff4-bda6-424bfd2f57eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:10.120997 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7867e-e620-4b68-a88d-df7baa2541a2 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:10.121089 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bb817-8153-4ff4-bda6-424bfd2f57eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:10.121736 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f7867e-e620-4b68-a88d-df7baa2541a2 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:10.122054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f7867e-e620-4b68-a88d-df7baa2541a2 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:10.123857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f7867e-e620-4b68-a88d-df7baa2541a2 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:10.125645 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2a7bb817-8153-4ff4-bda6-424bfd2f57eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:10.137692 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 932/1873] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:09:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:10.138974 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff65c3-ff0d-4ca2-8485-151a97cbd718 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:10.139277 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff65c3-ff0d-4ca2-8485-151a97cbd718 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:10.139567 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff65c3-ff0d-4ca2-8485-151a97cbd718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:10.157223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-18ff65c3-ff0d-4ca2-8485-151a97cbd718 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:10.158460 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-18ff65c3-ff0d-4ca2-8485-151a97cbd718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:10.184328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:10.191608 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7bb817-8153-4ff4-bda6-424bfd2f57eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:10.194245 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7bb817-8153-4ff4-bda6-424bfd2f57eb service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:10.212667 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 942/1874] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:10 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:10.214271 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bb817-8153-4ff4-bda6-424bfd2f57eb service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:10.215851 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2a7bb817-8153-4ff4-bda6-424bfd2f57eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:10.239216 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bb817-8153-4ff4-bda6-424bfd2f57eb service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:10.243515 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 933/1875] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:10.253047 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67453480-13ea-4b9a-8847-59c204ce68b7 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd0109feacf04f16838c357395098beb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac55065a0faa4689893dfceaa7807562', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:10.253256 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdecc0-6a80-48aa-ba32-1e381ed90ccb None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:10.253485 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdecc0-6a80-48aa-ba32-1e381ed90ccb None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:10.253739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bdecc0-6a80-48aa-ba32-1e381ed90ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:10.253938 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67453480-13ea-4b9a-8847-59c204ce68b7 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:10.254258 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67453480-13ea-4b9a-8847-59c204ce68b7 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:10.254575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67453480-13ea-4b9a-8847-59c204ce68b7 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:10.255316 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67453480-13ea-4b9a-8847-59c204ce68b7 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:10.257369 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67453480-13ea-4b9a-8847-59c204ce68b7 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:10.273613 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-82bdecc0-6a80-48aa-ba32-1e381ed90ccb None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:10.274907 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-82bdecc0-6a80-48aa-ba32-1e381ed90ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:10.298605 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 943/1876] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:10 2022] POST /identity/v3/projects => generated 354 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:10.300549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 934/1877] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:10.302449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bdecc0-6a80-48aa-ba32-1e381ed90ccb None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:10.306001 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67453480-13ea-4b9a-8847-59c204ce68b7 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:10.318713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bdecc0-6a80-48aa-ba32-1e381ed90ccb None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:10.414137 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67453480-13ea-4b9a-8847-59c204ce68b7 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:10.419699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cea23-4073-45e6-af7d-36c3bc3cdc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:10.421017 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cea23-4073-45e6-af7d-36c3bc3cdc3a admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:10.421313 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cea23-4073-45e6-af7d-36c3bc3cdc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:10.421713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cea23-4073-45e6-af7d-36c3bc3cdc3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:10.422591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153cea23-4073-45e6-af7d-36c3bc3cdc3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:10.423414 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153cea23-4073-45e6-af7d-36c3bc3cdc3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:10.425407 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153cea23-4073-45e6-af7d-36c3bc3cdc3a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:10.442128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 944/1878] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:10.449815 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cea23-4073-45e6-af7d-36c3bc3cdc3a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:10.477473 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cea23-4073-45e6-af7d-36c3bc3cdc3a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:10.479774 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0d910-9818-48c8-b9d9-dfbb4ad47a82 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd0109feacf04f16838c357395098beb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac55065a0faa4689893dfceaa7807562', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:10.481190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0d910-9818-48c8-b9d9-dfbb4ad47a82 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:10.481797 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0d910-9818-48c8-b9d9-dfbb4ad47a82 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:10.482261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0d910-9818-48c8-b9d9-dfbb4ad47a82 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:10.484091 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0d910-9818-48c8-b9d9-dfbb4ad47a82 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:10.485216 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0d910-9818-48c8-b9d9-dfbb4ad47a82 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:10.488556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1be62-cfdf-407e-9328-da0f5929d8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:10.489825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1be62-cfdf-407e-9328-da0f5929d8f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:10.490633 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0d910-9818-48c8-b9d9-dfbb4ad47a82 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:10.490942 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1be62-cfdf-407e-9328-da0f5929d8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:10.491457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1be62-cfdf-407e-9328-da0f5929d8f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:10.492464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1be62-cfdf-407e-9328-da0f5929d8f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:10.495211 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1be62-cfdf-407e-9328-da0f5929d8f3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:10.516163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 935/1879] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:10.547088 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 945/1880] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:10 2022] POST /identity/v3/projects => generated 391 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:10.551326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0d910-9818-48c8-b9d9-dfbb4ad47a82 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:10.566632 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0d910-9818-48c8-b9d9-dfbb4ad47a82 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:10.587920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1be62-cfdf-407e-9328-da0f5929d8f3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:10.657022 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:10.683521 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705351d-1056-4878-996c-ad7bc6935ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:10.684255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705351d-1056-4878-996c-ad7bc6935ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:10.684549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705351d-1056-4878-996c-ad7bc6935ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:10.684869 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705351d-1056-4878-996c-ad7bc6935ad6 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:10.685532 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705351d-1056-4878-996c-ad7bc6935ad6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:10.687485 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705351d-1056-4878-996c-ad7bc6935ad6 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:10.699577 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1be62-cfdf-407e-9328-da0f5929d8f3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:10.706711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616e4d5-bdd6-49ae-ba18-b97e439239c9 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd0109feacf04f16838c357395098beb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac55065a0faa4689893dfceaa7807562', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:10.707484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616e4d5-bdd6-49ae-ba18-b97e439239c9 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:10.707784 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616e4d5-bdd6-49ae-ba18-b97e439239c9 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:10.708487 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616e4d5-bdd6-49ae-ba18-b97e439239c9 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] PATH_INFO: `/v3/projects/cb2aef96f39d41758895a1b0279f4886` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:10.710273 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616e4d5-bdd6-49ae-ba18-b97e439239c9 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cb2aef96f39d41758895a1b0279f4886)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:10.721997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616e4d5-bdd6-49ae-ba18-b97e439239c9 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:10.757011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 936/1881] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:10 2022] POST /identity/v3/users => generated 296 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:10.759323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705351d-1056-4878-996c-ad7bc6935ad6 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:10.769875 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705351d-1056-4878-996c-ad7bc6935ad6 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:10.775442 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9efc50-f2d8-4371-9013-d736d7bb77d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:10.775915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9efc50-f2d8-4371-9013-d736d7bb77d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:10.776439 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9efc50-f2d8-4371-9013-d736d7bb77d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:10.776798 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9efc50-f2d8-4371-9013-d736d7bb77d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:10.777630 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9efc50-f2d8-4371-9013-d736d7bb77d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:10.779368 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9efc50-f2d8-4371-9013-d736d7bb77d1 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:10.795902 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 937/1882] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:09:10.802690 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9efc50-f2d8-4371-9013-d736d7bb77d1 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:10.805588 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8616e4d5-bdd6-49ae-ba18-b97e439239c9 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb2aef96f39d41758895a1b0279f4886', 'request_id': 'req-8616e4d5-bdd6-49ae-ba18-b97e439239c9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:10.812758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8616e4d5-bdd6-49ae-ba18-b97e439239c9 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb2aef96f39d41758895a1b0279f4886', 'request_id': 'req-8616e4d5-bdd6-49ae-ba18-b97e439239c9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:10.820176 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9efc50-f2d8-4371-9013-d736d7bb77d1 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:10.821333 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8616e4d5-bdd6-49ae-ba18-b97e439239c9 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb2aef96f39d41758895a1b0279f4886', 'request_id': 'req-8616e4d5-bdd6-49ae-ba18-b97e439239c9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:10.839868 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8616e4d5-bdd6-49ae-ba18-b97e439239c9 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] The token cache is being invalidate because project cb2aef96f39d41758895a1b0279f4886 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:10.840195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8616e4d5-bdd6-49ae-ba18-b97e439239c9 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:10.840976 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8616e4d5-bdd6-49ae-ba18-b97e439239c9 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:10.843983 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 946/1883] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:10 2022] DELETE /identity/v3/projects/cb2aef96f39d41758895a1b0279f4886 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:10.850677 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:10.893260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34698a85-b81f-4673-bead-1854c0a84fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:10.894099 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34698a85-b81f-4673-bead-1854c0a84fbd admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:10.894539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34698a85-b81f-4673-bead-1854c0a84fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:10.894827 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34698a85-b81f-4673-bead-1854c0a84fbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:10.895563 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34698a85-b81f-4673-bead-1854c0a84fbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:10.897404 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34698a85-b81f-4673-bead-1854c0a84fbd admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:10.912731 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 938/1884] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:09:10.913749 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616e4d5-bdd6-49ae-ba18-b97e439239c9 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:10.915394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8616e4d5-bdd6-49ae-ba18-b97e439239c9 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:10.933953 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616e4d5-bdd6-49ae-ba18-b97e439239c9 tempest-VolumeQuotaClassesTest-2027885875 tempest-VolumeQuotaClassesTest-2027885875-project-admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:10.944803 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558daf66-9d12-4b97-9a2f-c889cb00f450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:10.947533 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558daf66-9d12-4b97-9a2f-c889cb00f450 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:10.947918 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558daf66-9d12-4b97-9a2f-c889cb00f450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:10.948347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558daf66-9d12-4b97-9a2f-c889cb00f450 admin admin] PATH_INFO: `/v3/projects/35c3fcf9b8634dbd9ce799192830a1e2/users/31504804da20434cb86adae63b474371/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:10.967911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558daf66-9d12-4b97-9a2f-c889cb00f450 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35c3fcf9b8634dbd9ce799192830a1e2, user_id=31504804da20434cb86adae63b474371, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:10.971020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558daf66-9d12-4b97-9a2f-c889cb00f450 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:10.984873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 947/1885] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:10 2022] PUT /identity/v3/projects/35c3fcf9b8634dbd9ce799192830a1e2/users/31504804da20434cb86adae63b474371/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:10.996125 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d0ccd-0ff5-442b-a80b-08074de059af None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:10.996384 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d0ccd-0ff5-442b-a80b-08074de059af None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:10.996809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d0ccd-0ff5-442b-a80b-08074de059af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:11.029586 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-be2d0ccd-0ff5-442b-a80b-08074de059af None None] MFA Rules not processed for user `31504804da20434cb86adae63b474371`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:11.031271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-be2d0ccd-0ff5-442b-a80b-08074de059af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:11.054716 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558daf66-9d12-4b97-9a2f-c889cb00f450 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:11.057146 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-558daf66-9d12-4b97-9a2f-c889cb00f450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:11.115883 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 939/1886] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:11.115883 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d0ccd-0ff5-442b-a80b-08074de059af None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:11.115883 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-be2d0ccd-0ff5-442b-a80b-08074de059af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:11.133552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558daf66-9d12-4b97-9a2f-c889cb00f450 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:11.138595 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6542de-bcd6-4e97-bc7c-87e1f05fb511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:11.139496 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6542de-bcd6-4e97-bc7c-87e1f05fb511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:11.139905 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6542de-bcd6-4e97-bc7c-87e1f05fb511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:11.140207 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6542de-bcd6-4e97-bc7c-87e1f05fb511 admin admin] PATH_INFO: `/v3/users/8ff549a5cc874432ae5f0c2cfc9bd965` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:11.141412 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6542de-bcd6-4e97-bc7c-87e1f05fb511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ff549a5cc874432ae5f0c2cfc9bd965)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:11.143345 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6542de-bcd6-4e97-bc7c-87e1f05fb511 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:11.186189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d0ccd-0ff5-442b-a80b-08074de059af None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:11.190903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787330d-63dc-44b0-967c-d8dd7a880438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:11.191932 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:11.192065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787330d-63dc-44b0-967c-d8dd7a880438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:11.192355 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787330d-63dc-44b0-967c-d8dd7a880438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:11.192662 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787330d-63dc-44b0-967c-d8dd7a880438 admin admin] PATH_INFO: `/v3/users/8f3057b35f9d4089af66fab97610aab3` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:11.193993 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6e6542de-bcd6-4e97-bc7c-87e1f05fb511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ff549a5cc874432ae5f0c2cfc9bd965'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:11.194168 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5787330d-63dc-44b0-967c-d8dd7a880438 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f3057b35f9d4089af66fab97610aab3)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:11.196518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5787330d-63dc-44b0-967c-d8dd7a880438 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:11.202775 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6e6542de-bcd6-4e97-bc7c-87e1f05fb511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ff549a5cc874432ae5f0c2cfc9bd965'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:11.213371 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6e6542de-bcd6-4e97-bc7c-87e1f05fb511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ff549a5cc874432ae5f0c2cfc9bd965'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:11.214149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6e6542de-bcd6-4e97-bc7c-87e1f05fb511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ff549a5cc874432ae5f0c2cfc9bd965'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:11.231354 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6e6542de-bcd6-4e97-bc7c-87e1f05fb511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ff549a5cc874432ae5f0c2cfc9bd965'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:11.237767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 948/1887] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:11 2022] DELETE /identity/v3/users/8ff549a5cc874432ae5f0c2cfc9bd965 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:11.238632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6542de-bcd6-4e97-bc7c-87e1f05fb511 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:11.239964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6e6542de-bcd6-4e97-bc7c-87e1f05fb511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:11.248974 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5787330d-63dc-44b0-967c-d8dd7a880438 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f3057b35f9d4089af66fab97610aab3'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:11.252611 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:11.256823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5787330d-63dc-44b0-967c-d8dd7a880438 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f3057b35f9d4089af66fab97610aab3'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:11.266535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5787330d-63dc-44b0-967c-d8dd7a880438 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f3057b35f9d4089af66fab97610aab3'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:11.267220 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5787330d-63dc-44b0-967c-d8dd7a880438 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f3057b35f9d4089af66fab97610aab3'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:11.280510 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5787330d-63dc-44b0-967c-d8dd7a880438 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f3057b35f9d4089af66fab97610aab3'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:11.286752 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 940/1888] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:11 2022] DELETE /identity/v3/users/8f3057b35f9d4089af66fab97610aab3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:11.289896 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4b3da-51ad-4aa2-97a2-2dcf527a64ec None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:11.289896 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4b3da-51ad-4aa2-97a2-2dcf527a64ec None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:11.289896 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4b3da-51ad-4aa2-97a2-2dcf527a64ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:11.307611 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6542de-bcd6-4e97-bc7c-87e1f05fb511 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:11.307863 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f7a4b3da-51ad-4aa2-97a2-2dcf527a64ec None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:11.309142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f7a4b3da-51ad-4aa2-97a2-2dcf527a64ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:11.363927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a67fd-a491-453b-8e56-d45831deab33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:11.364630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a67fd-a491-453b-8e56-d45831deab33 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:11.364920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a67fd-a491-453b-8e56-d45831deab33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:11.365205 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a67fd-a491-453b-8e56-d45831deab33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:11.365908 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a67fd-a491-453b-8e56-d45831deab33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:11.368029 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a67fd-a491-453b-8e56-d45831deab33 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:11.369808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 941/1889] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:11.371081 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b5c78-7163-4148-a187-21cc48b88b99 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:11.371542 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b5c78-7163-4148-a187-21cc48b88b99 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:11.371833 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b5c78-7163-4148-a187-21cc48b88b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:11.390470 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b98b5c78-7163-4148-a187-21cc48b88b99 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:11.391721 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b98b5c78-7163-4148-a187-21cc48b88b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:11.398282 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 949/1890] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:11 2022] POST /identity/v3/projects => generated 391 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:11.398282 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a67fd-a491-453b-8e56-d45831deab33 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:11.398282 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a81a67fd-a491-453b-8e56-d45831deab33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:11.400468 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 942/1891] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:11.401229 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b5c78-7163-4148-a187-21cc48b88b99 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:11.402449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b98b5c78-7163-4148-a187-21cc48b88b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:11.416794 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b5c78-7163-4148-a187-21cc48b88b99 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:11.468017 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a67fd-a491-453b-8e56-d45831deab33 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:11.472099 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ad449-b311-4430-9a67-81792bfda860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:11.472797 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ad449-b311-4430-9a67-81792bfda860 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:11.473092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ad449-b311-4430-9a67-81792bfda860 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:11.473377 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ad449-b311-4430-9a67-81792bfda860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:11.483588 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:11.483877 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae315b7e-315d-423a-93bc-6d02a10175a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:11.484605 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae315b7e-315d-423a-93bc-6d02a10175a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:11.485058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae315b7e-315d-423a-93bc-6d02a10175a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:11.485500 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae315b7e-315d-423a-93bc-6d02a10175a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:11.486364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae315b7e-315d-423a-93bc-6d02a10175a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:11.488924 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae315b7e-315d-423a-93bc-6d02a10175a1 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:11.519537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 943/1892] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:11 2022] POST /identity/v3/users => generated 302 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:11.521861 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae315b7e-315d-423a-93bc-6d02a10175a1 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:11.523859 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ae315b7e-315d-423a-93bc-6d02a10175a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:11.544975 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae315b7e-315d-423a-93bc-6d02a10175a1 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:11.548940 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76fdf5-5525-4a4f-a37b-f9a69a94f3c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:11.549624 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76fdf5-5525-4a4f-a37b-f9a69a94f3c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:11.549975 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76fdf5-5525-4a4f-a37b-f9a69a94f3c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:11.550265 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76fdf5-5525-4a4f-a37b-f9a69a94f3c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:11.552443 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ad449-b311-4430-9a67-81792bfda860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:11.554005 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ad449-b311-4430-9a67-81792bfda860 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:11.563161 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76fdf5-5525-4a4f-a37b-f9a69a94f3c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:11.564675 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 950/1893] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:11.564787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76fdf5-5525-4a4f-a37b-f9a69a94f3c4 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:11.565320 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ad449-b311-4430-9a67-81792bfda860 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:11.575356 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ad449-b311-4430-9a67-81792bfda860 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:11.576822 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 944/1894] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:11.581308 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89101512-6356-414f-b259-a89a77f76d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:11.582000 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89101512-6356-414f-b259-a89a77f76d58 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:11.582323 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89101512-6356-414f-b259-a89a77f76d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:11.582618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89101512-6356-414f-b259-a89a77f76d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:11.583411 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89101512-6356-414f-b259-a89a77f76d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:11.585168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89101512-6356-414f-b259-a89a77f76d58 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:11.597279 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 951/1895] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:11.601033 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76fdf5-5525-4a4f-a37b-f9a69a94f3c4 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:11.613897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76fdf5-5525-4a4f-a37b-f9a69a94f3c4 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:11.619936 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967aa247-7072-4366-9417-7e0921459989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:11.620863 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967aa247-7072-4366-9417-7e0921459989 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:11.621156 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967aa247-7072-4366-9417-7e0921459989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:11.621466 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967aa247-7072-4366-9417-7e0921459989 admin admin] PATH_INFO: `/v3/projects/cf0f91c3ad3544b6b2d7e3c5e22eb8c3/users/cfa2bb47a4a54430b57900ec47b8330a/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:11.648382 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967aa247-7072-4366-9417-7e0921459989 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf0f91c3ad3544b6b2d7e3c5e22eb8c3, user_id=cfa2bb47a4a54430b57900ec47b8330a, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:11.651684 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967aa247-7072-4366-9417-7e0921459989 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:11.665872 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 945/1896] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:11 2022] PUT /identity/v3/projects/cf0f91c3ad3544b6b2d7e3c5e22eb8c3/users/cfa2bb47a4a54430b57900ec47b8330a/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:11.669688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89101512-6356-414f-b259-a89a77f76d58 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:11.682374 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89101512-6356-414f-b259-a89a77f76d58 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:11.836888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add2960-f87b-402c-8b0f-56c9bb1509d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:11.837604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add2960-f87b-402c-8b0f-56c9bb1509d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:11.837882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add2960-f87b-402c-8b0f-56c9bb1509d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:11.838674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add2960-f87b-402c-8b0f-56c9bb1509d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:11.839421 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add2960-f87b-402c-8b0f-56c9bb1509d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:11.849269 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add2960-f87b-402c-8b0f-56c9bb1509d8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:11.869036 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 952/1897] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:11.874051 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967aa247-7072-4366-9417-7e0921459989 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:11.886445 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967aa247-7072-4366-9417-7e0921459989 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:11.891633 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786df96-4f7c-405c-bc29-3fd318a94af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:11.892751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786df96-4f7c-405c-bc29-3fd318a94af7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:11.892922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786df96-4f7c-405c-bc29-3fd318a94af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:11.893244 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786df96-4f7c-405c-bc29-3fd318a94af7 admin admin] PATH_INFO: `/v3/projects/cf0f91c3ad3544b6b2d7e3c5e22eb8c3/users/cfa2bb47a4a54430b57900ec47b8330a/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:11.895689 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9786df96-4f7c-405c-bc29-3fd318a94af7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf0f91c3ad3544b6b2d7e3c5e22eb8c3, user_id=cfa2bb47a4a54430b57900ec47b8330a, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:11.898608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9786df96-4f7c-405c-bc29-3fd318a94af7 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:11.919986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 946/1898] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:11 2022] PUT /identity/v3/projects/cf0f91c3ad3544b6b2d7e3c5e22eb8c3/users/cfa2bb47a4a54430b57900ec47b8330a/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:11.928690 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea52cb2-64ff-4e18-9992-b834755acf11 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:11.928878 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea52cb2-64ff-4e18-9992-b834755acf11 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:11.929137 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea52cb2-64ff-4e18-9992-b834755acf11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:11.968139 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3ea52cb2-64ff-4e18-9992-b834755acf11 None None] MFA Rules not processed for user `cfa2bb47a4a54430b57900ec47b8330a`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:11.970065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3ea52cb2-64ff-4e18-9992-b834755acf11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:12.073786 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786df96-4f7c-405c-bc29-3fd318a94af7 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:12.078685 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9786df96-4f7c-405c-bc29-3fd318a94af7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:12.090131 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:12.100726 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 953/1899] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:11 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:12.112015 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea52cb2-64ff-4e18-9992-b834755acf11 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:12.115618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3ea52cb2-64ff-4e18-9992-b834755acf11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:12.158393 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:12.252755 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea52cb2-64ff-4e18-9992-b834755acf11 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:12.257785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7587c11-1c66-43e1-95d3-090eeb3d8efe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:12.259462 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7587c11-1c66-43e1-95d3-090eeb3d8efe service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:12.260190 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7587c11-1c66-43e1-95d3-090eeb3d8efe service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:12.260683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7587c11-1c66-43e1-95d3-090eeb3d8efe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:12.271190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786df96-4f7c-405c-bc29-3fd318a94af7 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:12.278990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7587c11-1c66-43e1-95d3-090eeb3d8efe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:12.280888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7587c11-1c66-43e1-95d3-090eeb3d8efe service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:12.282051 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602fc9c-9777-4a45-8b77-722414db5bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:12.283106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602fc9c-9777-4a45-8b77-722414db5bb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:12.283550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602fc9c-9777-4a45-8b77-722414db5bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:12.284049 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602fc9c-9777-4a45-8b77-722414db5bb0 admin admin] PATH_INFO: `/v3/projects/13901714ca504bd697a6807adbc38822` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:12.285420 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3602fc9c-9777-4a45-8b77-722414db5bb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13901714ca504bd697a6807adbc38822)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:12.288056 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3602fc9c-9777-4a45-8b77-722414db5bb0 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:12.302574 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 954/1900] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:12 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:12.303662 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7587c11-1c66-43e1-95d3-090eeb3d8efe service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:12.308415 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f7587c11-1c66-43e1-95d3-090eeb3d8efe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:12.334603 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7587c11-1c66-43e1-95d3-090eeb3d8efe service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:12.359064 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3602fc9c-9777-4a45-8b77-722414db5bb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13901714ca504bd697a6807adbc38822', 'request_id': 'req-3602fc9c-9777-4a45-8b77-722414db5bb0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:12.366932 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3602fc9c-9777-4a45-8b77-722414db5bb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13901714ca504bd697a6807adbc38822', 'request_id': 'req-3602fc9c-9777-4a45-8b77-722414db5bb0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:12.378824 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3602fc9c-9777-4a45-8b77-722414db5bb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13901714ca504bd697a6807adbc38822', 'request_id': 'req-3602fc9c-9777-4a45-8b77-722414db5bb0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:12.394305 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3602fc9c-9777-4a45-8b77-722414db5bb0 admin admin] The token cache is being invalidate because project 13901714ca504bd697a6807adbc38822 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:12.394611 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3602fc9c-9777-4a45-8b77-722414db5bb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:12.395290 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3602fc9c-9777-4a45-8b77-722414db5bb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:12.401022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 947/1901] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:12 2022] DELETE /identity/v3/projects/13901714ca504bd697a6807adbc38822 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:12.401927 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602fc9c-9777-4a45-8b77-722414db5bb0 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:12.411605 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3602fc9c-9777-4a45-8b77-722414db5bb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:12.427396 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455be7a-b35c-455c-beb8-8d3e4d960061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:12.428812 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455be7a-b35c-455c-beb8-8d3e4d960061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:12.429384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455be7a-b35c-455c-beb8-8d3e4d960061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:12.430643 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455be7a-b35c-455c-beb8-8d3e4d960061 admin admin] PATH_INFO: `/v3/projects/d8cbb59c7e944979ae1a83cf9395e8e8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:12.432242 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455be7a-b35c-455c-beb8-8d3e4d960061 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8cbb59c7e944979ae1a83cf9395e8e8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:12.434606 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455be7a-b35c-455c-beb8-8d3e4d960061 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:12.510953 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602fc9c-9777-4a45-8b77-722414db5bb0 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:12.518034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b455be7a-b35c-455c-beb8-8d3e4d960061 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8cbb59c7e944979ae1a83cf9395e8e8', 'request_id': 'req-b455be7a-b35c-455c-beb8-8d3e4d960061'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:12.525142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b455be7a-b35c-455c-beb8-8d3e4d960061 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8cbb59c7e944979ae1a83cf9395e8e8', 'request_id': 'req-b455be7a-b35c-455c-beb8-8d3e4d960061'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:12.532547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b455be7a-b35c-455c-beb8-8d3e4d960061 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8cbb59c7e944979ae1a83cf9395e8e8', 'request_id': 'req-b455be7a-b35c-455c-beb8-8d3e4d960061'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:12.544585 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b455be7a-b35c-455c-beb8-8d3e4d960061 admin admin] The token cache is being invalidate because project d8cbb59c7e944979ae1a83cf9395e8e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:12.544981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b455be7a-b35c-455c-beb8-8d3e4d960061 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:12.545655 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b455be7a-b35c-455c-beb8-8d3e4d960061 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:12.547504 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 955/1902] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:12 2022] DELETE /identity/v3/projects/d8cbb59c7e944979ae1a83cf9395e8e8 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:12.548125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455be7a-b35c-455c-beb8-8d3e4d960061 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:12.549414 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b455be7a-b35c-455c-beb8-8d3e4d960061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:12.562316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:12.568342 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4eaa2c-4794-42ac-bbbe-0140b8e9e2d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:12.569072 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4eaa2c-4794-42ac-bbbe-0140b8e9e2d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:12.569973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4eaa2c-4794-42ac-bbbe-0140b8e9e2d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:12.570327 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4eaa2c-4794-42ac-bbbe-0140b8e9e2d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:12.573095 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fc4eaa2c-4794-42ac-bbbe-0140b8e9e2d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:12.601560 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:12.619593 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455be7a-b35c-455c-beb8-8d3e4d960061 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:12.624986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240aba16-7621-4ab0-881f-1403d0fd9776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:12.625965 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240aba16-7621-4ab0-881f-1403d0fd9776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:12.626253 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240aba16-7621-4ab0-881f-1403d0fd9776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:12.626596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240aba16-7621-4ab0-881f-1403d0fd9776 admin admin] PATH_INFO: `/v3/users/a3648822de3442d888b658c538ae33a3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:12.628028 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240aba16-7621-4ab0-881f-1403d0fd9776 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3648822de3442d888b658c538ae33a3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:12.630040 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240aba16-7621-4ab0-881f-1403d0fd9776 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:12.636099 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4eaa2c-4794-42ac-bbbe-0140b8e9e2d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:12.637533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4eaa2c-4794-42ac-bbbe-0140b8e9e2d8 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:12.646905 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 948/1903] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:12.647541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4eaa2c-4794-42ac-bbbe-0140b8e9e2d8 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:12.648936 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fc4eaa2c-4794-42ac-bbbe-0140b8e9e2d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:12.663250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4eaa2c-4794-42ac-bbbe-0140b8e9e2d8 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:12.670698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45468c-dc9d-48f0-9f5d-75b87ee1ee70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:12.671973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45468c-dc9d-48f0-9f5d-75b87ee1ee70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:12.672387 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45468c-dc9d-48f0-9f5d-75b87ee1ee70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:12.672776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45468c-dc9d-48f0-9f5d-75b87ee1ee70 admin admin] PATH_INFO: `/v3/users/dd0109feacf04f16838c357395098beb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:12.674115 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45468c-dc9d-48f0-9f5d-75b87ee1ee70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd0109feacf04f16838c357395098beb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:12.677302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45468c-dc9d-48f0-9f5d-75b87ee1ee70 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:12.689733 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-240aba16-7621-4ab0-881f-1403d0fd9776 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3648822de3442d888b658c538ae33a3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:12.698352 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-240aba16-7621-4ab0-881f-1403d0fd9776 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3648822de3442d888b658c538ae33a3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:12.709679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-240aba16-7621-4ab0-881f-1403d0fd9776 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3648822de3442d888b658c538ae33a3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:12.711478 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-240aba16-7621-4ab0-881f-1403d0fd9776 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3648822de3442d888b658c538ae33a3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:12.746832 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4f45468c-dc9d-48f0-9f5d-75b87ee1ee70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd0109feacf04f16838c357395098beb'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:12.751451 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-240aba16-7621-4ab0-881f-1403d0fd9776 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3648822de3442d888b658c538ae33a3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:12.763207 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 956/1904] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:12 2022] DELETE /identity/v3/users/a3648822de3442d888b658c538ae33a3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:12.765273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4f45468c-dc9d-48f0-9f5d-75b87ee1ee70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd0109feacf04f16838c357395098beb'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:12.774536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4f45468c-dc9d-48f0-9f5d-75b87ee1ee70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd0109feacf04f16838c357395098beb'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:12.775274 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4f45468c-dc9d-48f0-9f5d-75b87ee1ee70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd0109feacf04f16838c357395098beb'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:12.790020 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4f45468c-dc9d-48f0-9f5d-75b87ee1ee70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd0109feacf04f16838c357395098beb'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:12.809099 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 949/1905] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:12 2022] DELETE /identity/v3/users/dd0109feacf04f16838c357395098beb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:13.644857 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240aba16-7621-4ab0-881f-1403d0fd9776 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:13.646589 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-240aba16-7621-4ab0-881f-1403d0fd9776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:13.659021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45468c-dc9d-48f0-9f5d-75b87ee1ee70 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:13.670762 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4f45468c-dc9d-48f0-9f5d-75b87ee1ee70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:13.759422 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240aba16-7621-4ab0-881f-1403d0fd9776 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:13.805279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45468c-dc9d-48f0-9f5d-75b87ee1ee70 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:13.813007 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99931e5-e2a1-43cc-a025-5fcce6efdfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:13.814299 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99931e5-e2a1-43cc-a025-5fcce6efdfb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:13.814596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99931e5-e2a1-43cc-a025-5fcce6efdfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:13.814885 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99931e5-e2a1-43cc-a025-5fcce6efdfb9 admin admin] PATH_INFO: `/v3/projects/99b98c1a3a7d4adfbab28961d5acd7a1` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:13.816499 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99931e5-e2a1-43cc-a025-5fcce6efdfb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99b98c1a3a7d4adfbab28961d5acd7a1)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:13.818898 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99931e5-e2a1-43cc-a025-5fcce6efdfb9 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:13.844859 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:13.851166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869c480-f8b3-4a87-8114-8df7135f273b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:13.852972 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869c480-f8b3-4a87-8114-8df7135f273b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:13.853237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869c480-f8b3-4a87-8114-8df7135f273b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:13.853627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869c480-f8b3-4a87-8114-8df7135f273b admin admin] PATH_INFO: `/v3/projects/ac55065a0faa4689893dfceaa7807562` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:13.858936 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869c480-f8b3-4a87-8114-8df7135f273b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac55065a0faa4689893dfceaa7807562)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:13.861644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869c480-f8b3-4a87-8114-8df7135f273b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:13.893294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a99931e5-e2a1-43cc-a025-5fcce6efdfb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99b98c1a3a7d4adfbab28961d5acd7a1', 'request_id': 'req-a99931e5-e2a1-43cc-a025-5fcce6efdfb9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:13.903422 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a99931e5-e2a1-43cc-a025-5fcce6efdfb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99b98c1a3a7d4adfbab28961d5acd7a1', 'request_id': 'req-a99931e5-e2a1-43cc-a025-5fcce6efdfb9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:13.904939 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:13.913597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a99931e5-e2a1-43cc-a025-5fcce6efdfb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99b98c1a3a7d4adfbab28961d5acd7a1', 'request_id': 'req-a99931e5-e2a1-43cc-a025-5fcce6efdfb9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:13.937624 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a99931e5-e2a1-43cc-a025-5fcce6efdfb9 admin admin] The token cache is being invalidate because project 99b98c1a3a7d4adfbab28961d5acd7a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:13.938269 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a99931e5-e2a1-43cc-a025-5fcce6efdfb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:13.941257 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a99931e5-e2a1-43cc-a025-5fcce6efdfb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:13.946749 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 950/1906] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:13 2022] DELETE /identity/v3/projects/99b98c1a3a7d4adfbab28961d5acd7a1 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:13.955646 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024068e-e460-490e-ab37-0799e405dd08 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:13.957806 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024068e-e460-490e-ab37-0799e405dd08 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:13.958098 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024068e-e460-490e-ab37-0799e405dd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:13.981471 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1869c480-f8b3-4a87-8114-8df7135f273b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac55065a0faa4689893dfceaa7807562', 'request_id': 'req-1869c480-f8b3-4a87-8114-8df7135f273b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:13.991673 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-6024068e-e460-490e-ab37-0799e405dd08 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:13.992992 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6024068e-e460-490e-ab37-0799e405dd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:13.997113 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1869c480-f8b3-4a87-8114-8df7135f273b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac55065a0faa4689893dfceaa7807562', 'request_id': 'req-1869c480-f8b3-4a87-8114-8df7135f273b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:14.011227 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1869c480-f8b3-4a87-8114-8df7135f273b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac55065a0faa4689893dfceaa7807562', 'request_id': 'req-1869c480-f8b3-4a87-8114-8df7135f273b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:14.037126 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1869c480-f8b3-4a87-8114-8df7135f273b admin admin] The token cache is being invalidate because project ac55065a0faa4689893dfceaa7807562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:14.038860 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1869c480-f8b3-4a87-8114-8df7135f273b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:14.040306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1869c480-f8b3-4a87-8114-8df7135f273b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:14.042986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 957/1907] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:13 2022] DELETE /identity/v3/projects/ac55065a0faa4689893dfceaa7807562 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:14.048212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0612a-6414-4656-9729-ff65fe391bac None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:14.048717 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0612a-6414-4656-9729-ff65fe391bac None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:14.049207 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0612a-6414-4656-9729-ff65fe391bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:14.073578 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-16b0612a-6414-4656-9729-ff65fe391bac None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:14.074839 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-16b0612a-6414-4656-9729-ff65fe391bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:14.153141 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 951/1908] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:09:14.161608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1549cc6-bb21-4b28-8eec-7048abe4331e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:14.161894 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1549cc6-bb21-4b28-8eec-7048abe4331e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:14.162308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1549cc6-bb21-4b28-8eec-7048abe4331e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:14.204231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d1549cc6-bb21-4b28-8eec-7048abe4331e None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:14.205223 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 958/1909] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:14.213050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d1549cc6-bb21-4b28-8eec-7048abe4331e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:14.213583 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11929f-3724-4a54-8ac1-cb3be2940c98 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:14.214179 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11929f-3724-4a54-8ac1-cb3be2940c98 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:14.214664 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc11929f-3724-4a54-8ac1-cb3be2940c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:14.226129 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 952/1910] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:09:14.231356 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1549cc6-bb21-4b28-8eec-7048abe4331e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:14.253626 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1549cc6-bb21-4b28-8eec-7048abe4331e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:14.266578 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0d2d8-49c5-4cc5-9cad-2d9eff4fbb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:14.268914 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0d2d8-49c5-4cc5-9cad-2d9eff4fbb9e admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:14.269872 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0d2d8-49c5-4cc5-9cad-2d9eff4fbb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:14.272188 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0d2d8-49c5-4cc5-9cad-2d9eff4fbb9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:14.273096 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0d2d8-49c5-4cc5-9cad-2d9eff4fbb9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:14.273487 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0d2d8-49c5-4cc5-9cad-2d9eff4fbb9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:14.278652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cc11929f-3724-4a54-8ac1-cb3be2940c98 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:14.278921 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0d2d8-49c5-4cc5-9cad-2d9eff4fbb9e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:14.279917 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cc11929f-3724-4a54-8ac1-cb3be2940c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:14.301772 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 953/1911] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:14.310916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0d2d8-49c5-4cc5-9cad-2d9eff4fbb9e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:14.314592 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 959/1912] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:09:14.316454 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11929f-3724-4a54-8ac1-cb3be2940c98 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:14.341518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0d2d8-49c5-4cc5-9cad-2d9eff4fbb9e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:14.352262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc11929f-3724-4a54-8ac1-cb3be2940c98 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:14.359395 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d09068-b355-451e-aef0-779c7d4b3785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:14.360047 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d09068-b355-451e-aef0-779c7d4b3785 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:14.360344 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d09068-b355-451e-aef0-779c7d4b3785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:14.360629 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d09068-b355-451e-aef0-779c7d4b3785 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:14.361382 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d09068-b355-451e-aef0-779c7d4b3785 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:14.362140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d09068-b355-451e-aef0-779c7d4b3785 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:14.363549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60c5c9-e0cb-46b4-800d-b0f1fb14c128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:14.364055 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d09068-b355-451e-aef0-779c7d4b3785 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:14.367678 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60c5c9-e0cb-46b4-800d-b0f1fb14c128 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:14.368219 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60c5c9-e0cb-46b4-800d-b0f1fb14c128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:14.369594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60c5c9-e0cb-46b4-800d-b0f1fb14c128 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:14.370297 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f60c5c9-e0cb-46b4-800d-b0f1fb14c128 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:14.376398 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f60c5c9-e0cb-46b4-800d-b0f1fb14c128 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:14.405368 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 960/1913] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:09:14.409585 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d09068-b355-451e-aef0-779c7d4b3785 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:14.422790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d09068-b355-451e-aef0-779c7d4b3785 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:14.432208 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874ac8c-31fa-4a7d-aa13-abf28d230a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:14.433377 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874ac8c-31fa-4a7d-aa13-abf28d230a0c admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:14.434155 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874ac8c-31fa-4a7d-aa13-abf28d230a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:14.434871 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c874ac8c-31fa-4a7d-aa13-abf28d230a0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:14.436237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874ac8c-31fa-4a7d-aa13-abf28d230a0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:14.439230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c874ac8c-31fa-4a7d-aa13-abf28d230a0c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:14.444979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 954/1914] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:14 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:14.455738 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60c5c9-e0cb-46b4-800d-b0f1fb14c128 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:14.495788 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 961/1915] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:14 2022] POST /identity/v3/projects => generated 387 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:14.499635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874ac8c-31fa-4a7d-aa13-abf28d230a0c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:14.615238 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c874ac8c-31fa-4a7d-aa13-abf28d230a0c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:14.630769 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028acca0-7e4d-40f7-9be4-e3ba70c2f6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:14.631882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028acca0-7e4d-40f7-9be4-e3ba70c2f6e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:14.633618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028acca0-7e4d-40f7-9be4-e3ba70c2f6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:14.634148 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028acca0-7e4d-40f7-9be4-e3ba70c2f6e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:14.635137 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028acca0-7e4d-40f7-9be4-e3ba70c2f6e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:14.639404 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028acca0-7e4d-40f7-9be4-e3ba70c2f6e1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:14.639750 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60c5c9-e0cb-46b4-800d-b0f1fb14c128 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:14.657516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927783e2-8cd8-4a78-ab2b-8765661c4902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:14.661654 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927783e2-8cd8-4a78-ab2b-8765661c4902 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:14.662229 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927783e2-8cd8-4a78-ab2b-8765661c4902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:14.662229 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927783e2-8cd8-4a78-ab2b-8765661c4902 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:14.663929 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:14.665443 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927783e2-8cd8-4a78-ab2b-8765661c4902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:14.670606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927783e2-8cd8-4a78-ab2b-8765661c4902 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:14.707007 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 962/1916] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:14 2022] POST /identity/v3/users => generated 294 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:09:14.709370 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028acca0-7e4d-40f7-9be4-e3ba70c2f6e1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:14.714529 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-028acca0-7e4d-40f7-9be4-e3ba70c2f6e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:14.729533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:14.756017 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028acca0-7e4d-40f7-9be4-e3ba70c2f6e1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:14.756221 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 955/1917] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:14 2022] POST /identity/v3/users => generated 290 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:14.757440 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927783e2-8cd8-4a78-ab2b-8765661c4902 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:14.782432 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0cadb-5719-4b06-aab4-3c2742c97373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:14.782432 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0cadb-5719-4b06-aab4-3c2742c97373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:14.782432 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0cadb-5719-4b06-aab4-3c2742c97373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:14.782432 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0cadb-5719-4b06-aab4-3c2742c97373 admin admin] PATH_INFO: `/v3/users/c5b42202b6dd4af084750ea3a025ae94` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:14.784083 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0cadb-5719-4b06-aab4-3c2742c97373 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5b42202b6dd4af084750ea3a025ae94)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:14.796766 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0cadb-5719-4b06-aab4-3c2742c97373 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:14.796967 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927783e2-8cd8-4a78-ab2b-8765661c4902 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:14.812989 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8367e-2dfa-4d03-8181-60b5fed401a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:14.813702 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8367e-2dfa-4d03-8181-60b5fed401a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:14.813998 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8367e-2dfa-4d03-8181-60b5fed401a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:14.817065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8367e-2dfa-4d03-8181-60b5fed401a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:14.817961 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8367e-2dfa-4d03-8181-60b5fed401a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:14.819643 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8367e-2dfa-4d03-8181-60b5fed401a7 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:14.849955 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 956/1918] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:14.853179 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8367e-2dfa-4d03-8181-60b5fed401a7 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:14.889559 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8367e-2dfa-4d03-8181-60b5fed401a7 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:14.907888 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511ad84-5247-4870-b3ed-2f84ecc50cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:14.909335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9fa0cadb-5719-4b06-aab4-3c2742c97373 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5b42202b6dd4af084750ea3a025ae94'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:14.909435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511ad84-5247-4870-b3ed-2f84ecc50cde admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:14.909435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511ad84-5247-4870-b3ed-2f84ecc50cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:14.909435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511ad84-5247-4870-b3ed-2f84ecc50cde admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:14.910293 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7511ad84-5247-4870-b3ed-2f84ecc50cde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:14.911847 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7511ad84-5247-4870-b3ed-2f84ecc50cde admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:14.925226 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9fa0cadb-5719-4b06-aab4-3c2742c97373 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5b42202b6dd4af084750ea3a025ae94'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:14.933825 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 957/1919] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:14.933825 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511ad84-5247-4870-b3ed-2f84ecc50cde admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:14.947768 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9fa0cadb-5719-4b06-aab4-3c2742c97373 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5b42202b6dd4af084750ea3a025ae94'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:14.948564 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9fa0cadb-5719-4b06-aab4-3c2742c97373 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5b42202b6dd4af084750ea3a025ae94'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:14.956416 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511ad84-5247-4870-b3ed-2f84ecc50cde admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:14.962202 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0211c77-e6f4-4a66-86ab-3cbfe67080ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:14.963026 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0211c77-e6f4-4a66-86ab-3cbfe67080ce admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:14.963394 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0211c77-e6f4-4a66-86ab-3cbfe67080ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:14.963751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0211c77-e6f4-4a66-86ab-3cbfe67080ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:14.964656 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0211c77-e6f4-4a66-86ab-3cbfe67080ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:14.966776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0211c77-e6f4-4a66-86ab-3cbfe67080ce admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:14.980237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 958/1920] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:14.981193 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0211c77-e6f4-4a66-86ab-3cbfe67080ce admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:14.981456 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9fa0cadb-5719-4b06-aab4-3c2742c97373 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5b42202b6dd4af084750ea3a025ae94'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:14.983675 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c0211c77-e6f4-4a66-86ab-3cbfe67080ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:14.995078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 963/1921] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:14 2022] DELETE /identity/v3/users/c5b42202b6dd4af084750ea3a025ae94 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:14.995078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0cadb-5719-4b06-aab4-3c2742c97373 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:14.995078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9fa0cadb-5719-4b06-aab4-3c2742c97373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:15.003474 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0211c77-e6f4-4a66-86ab-3cbfe67080ce admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:15.121890 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab81288-7615-4da5-82ab-e7fd6b48f9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:15.123341 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab81288-7615-4da5-82ab-e7fd6b48f9ca admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:15.123788 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab81288-7615-4da5-82ab-e7fd6b48f9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:15.124114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab81288-7615-4da5-82ab-e7fd6b48f9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:15.125412 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab81288-7615-4da5-82ab-e7fd6b48f9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:15.127511 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab81288-7615-4da5-82ab-e7fd6b48f9ca admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:15.146471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 959/1922] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:15.148120 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa62fe-3bff-4f79-b39f-83c30fe2da15 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:15.148454 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa62fe-3bff-4f79-b39f-83c30fe2da15 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:15.148873 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa62fe-3bff-4f79-b39f-83c30fe2da15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:15.163044 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0cadb-5719-4b06-aab4-3c2742c97373 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:15.173801 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-e9aa62fe-3bff-4f79-b39f-83c30fe2da15 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:15.175044 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e9aa62fe-3bff-4f79-b39f-83c30fe2da15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:15.176692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c557362-09f2-469a-bca6-1808d96b5930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:15.177959 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c557362-09f2-469a-bca6-1808d96b5930 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:15.179414 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c557362-09f2-469a-bca6-1808d96b5930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:15.180328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c557362-09f2-469a-bca6-1808d96b5930 admin admin] PATH_INFO: `/v3/projects/42e0babfb4c14e3a834da6b8fa45b9e9/users/e0936c8395d843db8392ca0ef7920e6f/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:15.188112 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 960/1923] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:15.192915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa62fe-3bff-4f79-b39f-83c30fe2da15 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:15.206584 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa62fe-3bff-4f79-b39f-83c30fe2da15 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:15.218325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8397d-c97b-4257-9f08-d987334d72da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:15.219236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8397d-c97b-4257-9f08-d987334d72da admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:15.219527 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8397d-c97b-4257-9f08-d987334d72da admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:15.219808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8397d-c97b-4257-9f08-d987334d72da admin admin] PATH_INFO: `/v3/projects/b828525ebf8c4afe8ad2e36944a6a0fd/users/1019d1ed18584d8f928230b54450377b/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:15.226452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c557362-09f2-469a-bca6-1808d96b5930 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e0babfb4c14e3a834da6b8fa45b9e9, user_id=e0936c8395d843db8392ca0ef7920e6f, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:15.230450 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c557362-09f2-469a-bca6-1808d96b5930 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:15.270503 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 964/1924] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:14 2022] PUT /identity/v3/projects/42e0babfb4c14e3a834da6b8fa45b9e9/users/e0936c8395d843db8392ca0ef7920e6f/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:15.271625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c557362-09f2-469a-bca6-1808d96b5930 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:15.274024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d8397d-c97b-4257-9f08-d987334d72da admin admin] RBAC: Authorizing `identity:create_grant(project_id=b828525ebf8c4afe8ad2e36944a6a0fd, user_id=1019d1ed18584d8f928230b54450377b, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:15.278780 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2c557362-09f2-469a-bca6-1808d96b5930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:15.286167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d8397d-c97b-4257-9f08-d987334d72da admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:15.333242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 961/1925] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:15 2022] PUT /identity/v3/projects/b828525ebf8c4afe8ad2e36944a6a0fd/users/1019d1ed18584d8f928230b54450377b/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:15.339154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817d7df-30de-4651-b53d-192b56a63ea9 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:15.340161 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817d7df-30de-4651-b53d-192b56a63ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:15.341675 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f817d7df-30de-4651-b53d-192b56a63ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:15.403317 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f817d7df-30de-4651-b53d-192b56a63ea9 None None] MFA Rules not processed for user `e0936c8395d843db8392ca0ef7920e6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:15.406448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f817d7df-30de-4651-b53d-192b56a63ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:15.438154 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:15.442721 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c557362-09f2-469a-bca6-1808d96b5930 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:15.532054 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20fc7a-1763-4003-bc1b-b52fd4e854af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:15.532054 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20fc7a-1763-4003-bc1b-b52fd4e854af service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:15.532354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 962/1926] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:15.532354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289b35d-b35c-4622-bc2b-2631cb2f112e None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:15.532491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20fc7a-1763-4003-bc1b-b52fd4e854af service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:15.532668 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20fc7a-1763-4003-bc1b-b52fd4e854af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:15.533537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289b35d-b35c-4622-bc2b-2631cb2f112e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:15.534609 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289b35d-b35c-4622-bc2b-2631cb2f112e None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:15.538475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 963/1927] 10.222.0.94 () {54 vars in 921 bytes} [Mon Jul 18 14:09:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:09:15.539841 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c96b7-cab3-4728-9087-173be917739c None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:15.542161 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c96b7-cab3-4728-9087-173be917739c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:15.542823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c96b7-cab3-4728-9087-173be917739c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:15.585128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-206c96b7-cab3-4728-9087-173be917739c None None] MFA Rules not processed for user `1019d1ed18584d8f928230b54450377b`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:15.587828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-206c96b7-cab3-4728-9087-173be917739c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:15.593537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:15.647166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20fc7a-1763-4003-bc1b-b52fd4e854af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:15.649533 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20fc7a-1763-4003-bc1b-b52fd4e854af service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:15.665662 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 965/1928] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:15.666504 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20fc7a-1763-4003-bc1b-b52fd4e854af service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:15.677859 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20fc7a-1763-4003-bc1b-b52fd4e854af service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:15.684572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1406a80-d445-4d89-b694-cc6b11e15131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:15.685409 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1406a80-d445-4d89-b694-cc6b11e15131 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:15.686006 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1406a80-d445-4d89-b694-cc6b11e15131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:15.686422 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1406a80-d445-4d89-b694-cc6b11e15131 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:15.687224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1406a80-d445-4d89-b694-cc6b11e15131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:15.690432 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1406a80-d445-4d89-b694-cc6b11e15131 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:15.712928 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 964/1929] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:15.715299 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e466502-e552-4367-9a7c-6810c43d24b0 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:15.715806 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e466502-e552-4367-9a7c-6810c43d24b0 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:15.716294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e466502-e552-4367-9a7c-6810c43d24b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:15.737912 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 966/1930] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:15 2022] POST /identity/v3/projects => generated 387 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:15.737912 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1406a80-d445-4d89-b694-cc6b11e15131 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:15.748834 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-7e466502-e552-4367-9a7c-6810c43d24b0 None None] MFA Rules not processed for user `ac717fcd9fc1499db12cf852d45572a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:15.750997 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7e466502-e552-4367-9a7c-6810c43d24b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:15.883563 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 965/1931] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Jul 18 14:09:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:15.884567 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e466502-e552-4367-9a7c-6810c43d24b0 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:15.889596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1406a80-d445-4d89-b694-cc6b11e15131 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:15.895257 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9f453-d907-47f1-9a64-c708b1f46a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:15.896611 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e466502-e552-4367-9a7c-6810c43d24b0 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:15.897064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9f453-d907-47f1-9a64-c708b1f46a8c admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:15.897421 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9f453-d907-47f1-9a64-c708b1f46a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:15.897966 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9f453-d907-47f1-9a64-c708b1f46a8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:15.898823 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a9f453-d907-47f1-9a64-c708b1f46a8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:15.901693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a9f453-d907-47f1-9a64-c708b1f46a8c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:15.902502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078bf076-7941-42f0-9f9f-51a505ab9e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:15.903928 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bf076-7941-42f0-9f9f-51a505ab9e8d admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:15.903928 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bf076-7941-42f0-9f9f-51a505ab9e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:15.904658 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bf076-7941-42f0-9f9f-51a505ab9e8d admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:15.906004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078bf076-7941-42f0-9f9f-51a505ab9e8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:15.908553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078bf076-7941-42f0-9f9f-51a505ab9e8d admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:15.950551 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 966/1932] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:15 2022] POST /identity/v3/users => generated 294 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:15.954886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078bf076-7941-42f0-9f9f-51a505ab9e8d admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:15.968565 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078bf076-7941-42f0-9f9f-51a505ab9e8d admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:15.980682 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 967/1933] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:15 2022] POST /identity/v3/projects => generated 379 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:15.983236 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9f453-d907-47f1-9a64-c708b1f46a8c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:15.987019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c0a9f453-d907-47f1-9a64-c708b1f46a8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:16.016371 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:16.107205 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29842e8d-f42a-4456-bfdf-e60bfdcd2fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:16.108823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29842e8d-f42a-4456-bfdf-e60bfdcd2fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:16.109394 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29842e8d-f42a-4456-bfdf-e60bfdcd2fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:16.109487 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29842e8d-f42a-4456-bfdf-e60bfdcd2fb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:16.110180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29842e8d-f42a-4456-bfdf-e60bfdcd2fb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:16.110870 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9f453-d907-47f1-9a64-c708b1f46a8c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:16.113761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29842e8d-f42a-4456-bfdf-e60bfdcd2fb3 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:16.118637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f7075-98db-46f3-bba2-fcdaf028f4ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:16.119421 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f7075-98db-46f3-bba2-fcdaf028f4ca service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:16.119783 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f7075-98db-46f3-bba2-fcdaf028f4ca service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:16.120164 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f7075-98db-46f3-bba2-fcdaf028f4ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:16.129312 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 967/1934] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:16.129312 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29842e8d-f42a-4456-bfdf-e60bfdcd2fb3 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:16.137174 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f7075-98db-46f3-bba2-fcdaf028f4ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:16.139791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f7075-98db-46f3-bba2-fcdaf028f4ca service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:16.147450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29842e8d-f42a-4456-bfdf-e60bfdcd2fb3 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:16.153743 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 968/1935] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:16.154773 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fceed-e564-41a9-b1dc-2f5f14cf024a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:16.154977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f7075-98db-46f3-bba2-fcdaf028f4ca service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:16.155610 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fceed-e564-41a9-b1dc-2f5f14cf024a admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:16.155991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fceed-e564-41a9-b1dc-2f5f14cf024a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:16.156547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fceed-e564-41a9-b1dc-2f5f14cf024a admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:16.160587 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21fceed-e564-41a9-b1dc-2f5f14cf024a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:16.167542 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21fceed-e564-41a9-b1dc-2f5f14cf024a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:16.172406 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f7075-98db-46f3-bba2-fcdaf028f4ca service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:16.178603 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293992ad-c635-4cb0-9664-c85a0eefc55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:16.179272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293992ad-c635-4cb0-9664-c85a0eefc55f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:16.189813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:16.195170 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293992ad-c635-4cb0-9664-c85a0eefc55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:16.195635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293992ad-c635-4cb0-9664-c85a0eefc55f admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:16.196477 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293992ad-c635-4cb0-9664-c85a0eefc55f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:16.198644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293992ad-c635-4cb0-9664-c85a0eefc55f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:16.202423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 968/1936] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:16 2022] POST /identity/v3/users => generated 296 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:16.203333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fceed-e564-41a9-b1dc-2f5f14cf024a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:16.205730 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a21fceed-e564-41a9-b1dc-2f5f14cf024a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:16.220800 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 969/1937] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:16.220800 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293992ad-c635-4cb0-9664-c85a0eefc55f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:16.221854 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21fceed-e564-41a9-b1dc-2f5f14cf024a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:16.227473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb9358-8b19-408b-94fc-b8131a9c7022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:16.229329 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb9358-8b19-408b-94fc-b8131a9c7022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:16.229634 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb9358-8b19-408b-94fc-b8131a9c7022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:16.230755 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb9358-8b19-408b-94fc-b8131a9c7022 admin admin] PATH_INFO: `/v3/projects/0b142446207e413ab254134f2aa4c48d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:16.232213 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fb9358-8b19-408b-94fc-b8131a9c7022 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b142446207e413ab254134f2aa4c48d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:16.234144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fb9358-8b19-408b-94fc-b8131a9c7022 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:16.249386 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293992ad-c635-4cb0-9664-c85a0eefc55f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:16.256548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9986116-c382-404a-b21b-f78117cadaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:16.257832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9986116-c382-404a-b21b-f78117cadaa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:16.258832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9986116-c382-404a-b21b-f78117cadaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:16.259129 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9986116-c382-404a-b21b-f78117cadaa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:16.259871 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9986116-c382-404a-b21b-f78117cadaa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:16.261573 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9986116-c382-404a-b21b-f78117cadaa1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:16.275082 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 970/1938] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:16.275082 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9986116-c382-404a-b21b-f78117cadaa1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:16.294013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9986116-c382-404a-b21b-f78117cadaa1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:16.303647 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa9efb-3842-41d4-b65f-8a7748380ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:16.305736 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa9efb-3842-41d4-b65f-8a7748380ba4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:16.306478 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa9efb-3842-41d4-b65f-8a7748380ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:16.306988 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa9efb-3842-41d4-b65f-8a7748380ba4 admin admin] PATH_INFO: `/v3/projects/a66d037cd93541a68bd32e178c3448eb/users/438b0bebb2634efca6f2e0c537a97786/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:16.319834 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-99fb9358-8b19-408b-94fc-b8131a9c7022 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b142446207e413ab254134f2aa4c48d', 'request_id': 'req-99fb9358-8b19-408b-94fc-b8131a9c7022'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:16.327247 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-99fb9358-8b19-408b-94fc-b8131a9c7022 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b142446207e413ab254134f2aa4c48d', 'request_id': 'req-99fb9358-8b19-408b-94fc-b8131a9c7022'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:16.338858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa9efb-3842-41d4-b65f-8a7748380ba4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a66d037cd93541a68bd32e178c3448eb, user_id=438b0bebb2634efca6f2e0c537a97786, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:16.339054 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-99fb9358-8b19-408b-94fc-b8131a9c7022 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b142446207e413ab254134f2aa4c48d', 'request_id': 'req-99fb9358-8b19-408b-94fc-b8131a9c7022'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:16.344007 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa9efb-3842-41d4-b65f-8a7748380ba4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:16.357988 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-99fb9358-8b19-408b-94fc-b8131a9c7022 admin admin] The token cache is being invalidate because project 0b142446207e413ab254134f2aa4c48d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:16.358342 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-99fb9358-8b19-408b-94fc-b8131a9c7022 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:16.359308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-99fb9358-8b19-408b-94fc-b8131a9c7022 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:16.367259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 969/1940] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:16 2022] DELETE /identity/v3/projects/0b142446207e413ab254134f2aa4c48d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:16.368294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb9358-8b19-408b-94fc-b8131a9c7022 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:16.369936 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-99fb9358-8b19-408b-94fc-b8131a9c7022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:16.373805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 971/1940] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:16 2022] PUT /identity/v3/projects/a66d037cd93541a68bd32e178c3448eb/users/438b0bebb2634efca6f2e0c537a97786/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:16.374836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa9efb-3842-41d4-b65f-8a7748380ba4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:16.376735 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-35fa9efb-3842-41d4-b65f-8a7748380ba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:16.495331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb9358-8b19-408b-94fc-b8131a9c7022 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:16.524804 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa9efb-3842-41d4-b65f-8a7748380ba4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:16.531351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71492398-0140-4289-a7dd-4bb39a975fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:16.532397 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71492398-0140-4289-a7dd-4bb39a975fea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:16.532759 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71492398-0140-4289-a7dd-4bb39a975fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:16.533226 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71492398-0140-4289-a7dd-4bb39a975fea admin admin] PATH_INFO: `/v3/users/941a58530932444b8cc9815783f1f8db` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:16.535818 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71492398-0140-4289-a7dd-4bb39a975fea admin admin] RBAC: Authorizing `identity:delete_user(user_id=941a58530932444b8cc9815783f1f8db)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:16.538595 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71492398-0140-4289-a7dd-4bb39a975fea admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:16.596127 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64531b-3ddd-47ce-b158-9564d466bf9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:16.597841 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64531b-3ddd-47ce-b158-9564d466bf9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:16.598342 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64531b-3ddd-47ce-b158-9564d466bf9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:16.599547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64531b-3ddd-47ce-b158-9564d466bf9f admin admin] PATH_INFO: `/v3/projects/c2aaa5ed1bd6459ba22188a1b7e34b5c/users/c69945b9919d49d69888168368d48377/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:16.609707 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-71492398-0140-4289-a7dd-4bb39a975fea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '941a58530932444b8cc9815783f1f8db'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:16.622054 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-71492398-0140-4289-a7dd-4bb39a975fea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '941a58530932444b8cc9815783f1f8db'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:16.628822 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:16.631174 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64531b-3ddd-47ce-b158-9564d466bf9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2aaa5ed1bd6459ba22188a1b7e34b5c, user_id=c69945b9919d49d69888168368d48377, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:16.632918 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-71492398-0140-4289-a7dd-4bb39a975fea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '941a58530932444b8cc9815783f1f8db'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:16.634154 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-71492398-0140-4289-a7dd-4bb39a975fea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '941a58530932444b8cc9815783f1f8db'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:16.636178 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64531b-3ddd-47ce-b158-9564d466bf9f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:16.658885 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 970/1941] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:16 2022] PUT /identity/v3/projects/c2aaa5ed1bd6459ba22188a1b7e34b5c/users/c69945b9919d49d69888168368d48377/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:16.660528 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d37551-4361-496e-a750-8426d91714ae None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:16.660951 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d37551-4361-496e-a750-8426d91714ae None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:16.661305 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d37551-4361-496e-a750-8426d91714ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:16.665489 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-71492398-0140-4289-a7dd-4bb39a975fea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '941a58530932444b8cc9815783f1f8db'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:16.680000 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 972/1942] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:16 2022] DELETE /identity/v3/users/941a58530932444b8cc9815783f1f8db => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:16.683652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71492398-0140-4289-a7dd-4bb39a975fea admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:16.683790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:16.687924 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-71492398-0140-4289-a7dd-4bb39a975fea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:16.697343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d9d37551-4361-496e-a750-8426d91714ae None None] MFA Rules not processed for user `438b0bebb2634efca6f2e0c537a97786`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:16.698623 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d9d37551-4361-496e-a750-8426d91714ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:16.839579 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 971/1943] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:16.843861 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d37551-4361-496e-a750-8426d91714ae None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:16.845802 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d9d37551-4361-496e-a750-8426d91714ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:16.855653 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71492398-0140-4289-a7dd-4bb39a975fea admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:16.863392 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d37551-4361-496e-a750-8426d91714ae None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:16.870276 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232a303-5d7d-44d6-9b4b-4d859ede8676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:16.870983 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232a303-5d7d-44d6-9b4b-4d859ede8676 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:16.871328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232a303-5d7d-44d6-9b4b-4d859ede8676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:16.871609 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232a303-5d7d-44d6-9b4b-4d859ede8676 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:16.872551 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232a303-5d7d-44d6-9b4b-4d859ede8676 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:16.875224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4062ed-9b89-4722-b8d6-7874565f4d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:16.875381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232a303-5d7d-44d6-9b4b-4d859ede8676 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:16.876503 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4062ed-9b89-4722-b8d6-7874565f4d90 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:16.878372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4062ed-9b89-4722-b8d6-7874565f4d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:16.878799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4062ed-9b89-4722-b8d6-7874565f4d90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:16.880737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4062ed-9b89-4722-b8d6-7874565f4d90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:16.886323 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4062ed-9b89-4722-b8d6-7874565f4d90 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:16.919967 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 973/1944] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:16.924599 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4062ed-9b89-4722-b8d6-7874565f4d90 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:16.949844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 972/1945] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:16 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:16.950997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4062ed-9b89-4722-b8d6-7874565f4d90 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:16.954952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232a303-5d7d-44d6-9b4b-4d859ede8676 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:17.094327 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232a303-5d7d-44d6-9b4b-4d859ede8676 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:17.099966 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ac47f-813c-4014-9c0a-b7d2898ca721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:17.101025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ac47f-813c-4014-9c0a-b7d2898ca721 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:17.101957 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ac47f-813c-4014-9c0a-b7d2898ca721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:17.102268 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ac47f-813c-4014-9c0a-b7d2898ca721 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:17.103043 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ac47f-813c-4014-9c0a-b7d2898ca721 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:17.105377 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ac47f-813c-4014-9c0a-b7d2898ca721 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:17.125791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8fda7-c018-40d6-bf58-35702a655175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:17.127072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8fda7-c018-40d6-bf58-35702a655175 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:17.127600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8fda7-c018-40d6-bf58-35702a655175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:17.127972 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8fda7-c018-40d6-bf58-35702a655175 admin admin] PATH_INFO: `/v3/projects/c2aaa5ed1bd6459ba22188a1b7e34b5c/users/c69945b9919d49d69888168368d48377/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:17.132035 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8fda7-c018-40d6-bf58-35702a655175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2aaa5ed1bd6459ba22188a1b7e34b5c, user_id=c69945b9919d49d69888168368d48377, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:17.135519 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8fda7-c018-40d6-bf58-35702a655175 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:17.149221 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 973/1946] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:16 2022] POST /identity/v3/users => generated 300 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:17.151916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ac47f-813c-4014-9c0a-b7d2898ca721 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:17.174722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 974/1947] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:16 2022] PUT /identity/v3/projects/c2aaa5ed1bd6459ba22188a1b7e34b5c/users/c69945b9919d49d69888168368d48377/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:17.178042 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399511f7-5025-4d90-b2bc-78bfb65f3f85 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:17.178401 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399511f7-5025-4d90-b2bc-78bfb65f3f85 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:17.178738 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399511f7-5025-4d90-b2bc-78bfb65f3f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:17.197403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ac47f-813c-4014-9c0a-b7d2898ca721 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:17.240676 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-399511f7-5025-4d90-b2bc-78bfb65f3f85 None None] MFA Rules not processed for user `c69945b9919d49d69888168368d48377`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:17.242833 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-399511f7-5025-4d90-b2bc-78bfb65f3f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:17.340138 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af01cba-d6a9-4055-846b-8a5fc2d7533b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:17.340974 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af01cba-d6a9-4055-846b-8a5fc2d7533b admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:17.341374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af01cba-d6a9-4055-846b-8a5fc2d7533b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:17.341755 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af01cba-d6a9-4055-846b-8a5fc2d7533b admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:17.342604 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af01cba-d6a9-4055-846b-8a5fc2d7533b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:17.344068 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:17.344532 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af01cba-d6a9-4055-846b-8a5fc2d7533b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:17.356723 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 974/1948] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:17.357698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af01cba-d6a9-4055-846b-8a5fc2d7533b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:17.359457 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5af01cba-d6a9-4055-846b-8a5fc2d7533b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:17.371469 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:17.376855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 975/1949] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:17 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:09:17.389414 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399511f7-5025-4d90-b2bc-78bfb65f3f85 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:17.391609 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-399511f7-5025-4d90-b2bc-78bfb65f3f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:17.396406 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af01cba-d6a9-4055-846b-8a5fc2d7533b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:17.408792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399511f7-5025-4d90-b2bc-78bfb65f3f85 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:17.409817 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04e786-4863-40f1-9220-05032cb170fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:17.411523 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04e786-4863-40f1-9220-05032cb170fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:17.412053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04e786-4863-40f1-9220-05032cb170fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:17.412540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04e786-4863-40f1-9220-05032cb170fb admin admin] PATH_INFO: `/v3/users/31504804da20434cb86adae63b474371` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:17.414986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad04e786-4863-40f1-9220-05032cb170fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=31504804da20434cb86adae63b474371)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:17.416136 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4832a-96f3-4471-8e99-8c3c3f997e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:17.417032 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4832a-96f3-4471-8e99-8c3c3f997e1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:17.417317 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4832a-96f3-4471-8e99-8c3c3f997e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:17.417599 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4832a-96f3-4471-8e99-8c3c3f997e1f admin admin] PATH_INFO: `/v3/projects/2894e0d6363f42a085257394cb6dd0bc` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:17.418187 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad04e786-4863-40f1-9220-05032cb170fb admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:17.418862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da4832a-96f3-4471-8e99-8c3c3f997e1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2894e0d6363f42a085257394cb6dd0bc)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:17.421026 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da4832a-96f3-4471-8e99-8c3c3f997e1f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:17.486313 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6da4832a-96f3-4471-8e99-8c3c3f997e1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2894e0d6363f42a085257394cb6dd0bc', 'request_id': 'req-6da4832a-96f3-4471-8e99-8c3c3f997e1f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:17.489669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ad04e786-4863-40f1-9220-05032cb170fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31504804da20434cb86adae63b474371'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:17.493428 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6da4832a-96f3-4471-8e99-8c3c3f997e1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2894e0d6363f42a085257394cb6dd0bc', 'request_id': 'req-6da4832a-96f3-4471-8e99-8c3c3f997e1f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:17.498818 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ad04e786-4863-40f1-9220-05032cb170fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31504804da20434cb86adae63b474371'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:17.501847 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6da4832a-96f3-4471-8e99-8c3c3f997e1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2894e0d6363f42a085257394cb6dd0bc', 'request_id': 'req-6da4832a-96f3-4471-8e99-8c3c3f997e1f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:17.508612 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ad04e786-4863-40f1-9220-05032cb170fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31504804da20434cb86adae63b474371'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:17.509854 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ad04e786-4863-40f1-9220-05032cb170fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31504804da20434cb86adae63b474371'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:17.513819 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6da4832a-96f3-4471-8e99-8c3c3f997e1f admin admin] The token cache is being invalidate because project 2894e0d6363f42a085257394cb6dd0bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:17.514149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6da4832a-96f3-4471-8e99-8c3c3f997e1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:17.514783 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6da4832a-96f3-4471-8e99-8c3c3f997e1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:17.516610 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 976/1950] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:17 2022] DELETE /identity/v3/projects/2894e0d6363f42a085257394cb6dd0bc => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:17.517149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4832a-96f3-4471-8e99-8c3c3f997e1f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:17.521765 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6da4832a-96f3-4471-8e99-8c3c3f997e1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:17.529512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ad04e786-4863-40f1-9220-05032cb170fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31504804da20434cb86adae63b474371'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:17.537363 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 975/1951] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:17 2022] DELETE /identity/v3/users/31504804da20434cb86adae63b474371 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:17.538442 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04e786-4863-40f1-9220-05032cb170fb admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:17.540871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ad04e786-4863-40f1-9220-05032cb170fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:17.598102 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4832a-96f3-4471-8e99-8c3c3f997e1f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:17.609089 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04e786-4863-40f1-9220-05032cb170fb admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:17.613533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8ca1f-239f-4a24-b419-4b283b25271f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:17.614268 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8ca1f-239f-4a24-b419-4b283b25271f service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:17.614626 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8ca1f-239f-4a24-b419-4b283b25271f service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:17.615006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8ca1f-239f-4a24-b419-4b283b25271f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:17.616946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8cb8ca1f-239f-4a24-b419-4b283b25271f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:17.670732 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a4f2c-eabb-4037-91f1-c31a8e68946f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:17.672188 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a4f2c-eabb-4037-91f1-c31a8e68946f admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:17.672583 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a4f2c-eabb-4037-91f1-c31a8e68946f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:17.673605 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a4f2c-eabb-4037-91f1-c31a8e68946f admin admin] PATH_INFO: `/v3/projects/cb688abaca0a4e6d8a57334e3d37a285/users/af3fd7c71ee74e0b94d5b2bfc84cb69d/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:17.688718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8ca1f-239f-4a24-b419-4b283b25271f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:17.690267 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8ca1f-239f-4a24-b419-4b283b25271f service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:17.690415 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a4f2c-eabb-4037-91f1-c31a8e68946f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb688abaca0a4e6d8a57334e3d37a285, user_id=af3fd7c71ee74e0b94d5b2bfc84cb69d, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:17.693690 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a4f2c-eabb-4037-91f1-c31a8e68946f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:17.702538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 976/1952] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:17 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:17.704644 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0c3c4-11dc-4b7f-8fbb-1bd51fd23989 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:17.704953 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0c3c4-11dc-4b7f-8fbb-1bd51fd23989 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:17.705359 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0c3c4-11dc-4b7f-8fbb-1bd51fd23989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:17.716920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 977/1953] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:17 2022] PUT /identity/v3/projects/cb688abaca0a4e6d8a57334e3d37a285/users/af3fd7c71ee74e0b94d5b2bfc84cb69d/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:17.718414 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea330173-36a2-4759-99cd-a73d765dff71 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:17.719139 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea330173-36a2-4759-99cd-a73d765dff71 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:17.719491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea330173-36a2-4759-99cd-a73d765dff71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:17.729461 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:17.732232 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-e5a0c3c4-11dc-4b7f-8fbb-1bd51fd23989 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:17.733950 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e5a0c3c4-11dc-4b7f-8fbb-1bd51fd23989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:17.747013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ea330173-36a2-4759-99cd-a73d765dff71 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:17.749473 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ea330173-36a2-4759-99cd-a73d765dff71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:17.771488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:17.880738 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 977/1954] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:17.887562 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0c3c4-11dc-4b7f-8fbb-1bd51fd23989 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:17.906658 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 978/1955] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:17.908086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858f989-b1b7-47af-8420-535f42144f19 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:17.908857 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858f989-b1b7-47af-8420-535f42144f19 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:17.908857 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858f989-b1b7-47af-8420-535f42144f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:17.923982 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0c3c4-11dc-4b7f-8fbb-1bd51fd23989 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:17.934961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d858f989-b1b7-47af-8420-535f42144f19 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:17.934961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d858f989-b1b7-47af-8420-535f42144f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:17.948203 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 979/1956] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:17.948796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858f989-b1b7-47af-8420-535f42144f19 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:17.950581 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d858f989-b1b7-47af-8420-535f42144f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:17.966737 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc8d42-e380-4fdd-a126-c2968982f947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:17.967459 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc8d42-e380-4fdd-a126-c2968982f947 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:17.967755 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc8d42-e380-4fdd-a126-c2968982f947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:17.968072 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc8d42-e380-4fdd-a126-c2968982f947 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:17.968825 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc8d42-e380-4fdd-a126-c2968982f947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:17.970960 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc8d42-e380-4fdd-a126-c2968982f947 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:18.003458 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 978/1957] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:18.004376 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc8d42-e380-4fdd-a126-c2968982f947 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:18.020086 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc8d42-e380-4fdd-a126-c2968982f947 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:18.037610 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a860a6-7db4-4d03-8530-b6bf85573a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:18.038293 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a860a6-7db4-4d03-8530-b6bf85573a81 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:18.038588 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a860a6-7db4-4d03-8530-b6bf85573a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:18.039398 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a860a6-7db4-4d03-8530-b6bf85573a81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:18.040217 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a860a6-7db4-4d03-8530-b6bf85573a81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:18.040476 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a860a6-7db4-4d03-8530-b6bf85573a81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:18.042093 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a860a6-7db4-4d03-8530-b6bf85573a81 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:18.050927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858f989-b1b7-47af-8420-535f42144f19 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:18.055808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc3df7-fcb5-4bdd-b9ed-b437f5e6aedd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:18.056938 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc3df7-fcb5-4bdd-b9ed-b437f5e6aedd service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:18.061177 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 979/1958] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:18.062170 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a860a6-7db4-4d03-8530-b6bf85573a81 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:18.062291 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc3df7-fcb5-4bdd-b9ed-b437f5e6aedd service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:18.062519 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc3df7-fcb5-4bdd-b9ed-b437f5e6aedd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:18.076109 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a860a6-7db4-4d03-8530-b6bf85573a81 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:18.077675 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc3df7-fcb5-4bdd-b9ed-b437f5e6aedd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:18.081039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc3df7-fcb5-4bdd-b9ed-b437f5e6aedd service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:18.082918 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53395d69-d389-4bf6-8e3c-7aae5bbb0757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:18.084979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53395d69-d389-4bf6-8e3c-7aae5bbb0757 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:18.085537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53395d69-d389-4bf6-8e3c-7aae5bbb0757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:18.086001 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53395d69-d389-4bf6-8e3c-7aae5bbb0757 admin admin] PATH_INFO: `/v3/projects/cb688abaca0a4e6d8a57334e3d37a285/users/af3fd7c71ee74e0b94d5b2bfc84cb69d/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:18.088447 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53395d69-d389-4bf6-8e3c-7aae5bbb0757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb688abaca0a4e6d8a57334e3d37a285, user_id=af3fd7c71ee74e0b94d5b2bfc84cb69d, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:18.093412 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53395d69-d389-4bf6-8e3c-7aae5bbb0757 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:18.096472 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 980/1959] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:18.099534 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc3df7-fcb5-4bdd-b9ed-b437f5e6aedd service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:18.119191 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc3df7-fcb5-4bdd-b9ed-b437f5e6aedd service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:18.123129 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 980/1960] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:18 2022] PUT /identity/v3/projects/cb688abaca0a4e6d8a57334e3d37a285/users/af3fd7c71ee74e0b94d5b2bfc84cb69d/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:18.134775 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c276c-7eef-48c1-9487-b598f312801a None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:18.135406 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c276c-7eef-48c1-9487-b598f312801a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:18.135553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c276c-7eef-48c1-9487-b598f312801a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:18.199268 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-bd4c276c-7eef-48c1-9487-b598f312801a None None] MFA Rules not processed for user `af3fd7c71ee74e0b94d5b2bfc84cb69d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:18.200509 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bd4c276c-7eef-48c1-9487-b598f312801a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:18.265186 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065aaa2-d53b-4b90-978c-4dafa0b5ec93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:18.265893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065aaa2-d53b-4b90-978c-4dafa0b5ec93 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:18.266158 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065aaa2-d53b-4b90-978c-4dafa0b5ec93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:18.266442 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065aaa2-d53b-4b90-978c-4dafa0b5ec93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:18.267739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065aaa2-d53b-4b90-978c-4dafa0b5ec93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:18.269770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065aaa2-d53b-4b90-978c-4dafa0b5ec93 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:18.313173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 981/1961] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:18 2022] POST /identity/v3/projects => generated 377 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:18.314667 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065aaa2-d53b-4b90-978c-4dafa0b5ec93 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:18.360591 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 981/1962] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:18.361675 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c276c-7eef-48c1-9487-b598f312801a None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:18.366477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bd4c276c-7eef-48c1-9487-b598f312801a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:18.372339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:18.383920 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c276c-7eef-48c1-9487-b598f312801a None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:18.418486 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065aaa2-d53b-4b90-978c-4dafa0b5ec93 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:18.423983 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fdc25-8d92-4799-ba75-e394a4e16752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:18.424140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fdc25-8d92-4799-ba75-e394a4e16752 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:18.424437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fdc25-8d92-4799-ba75-e394a4e16752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:18.424755 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fdc25-8d92-4799-ba75-e394a4e16752 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:18.425664 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47fdc25-8d92-4799-ba75-e394a4e16752 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:18.427734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47fdc25-8d92-4799-ba75-e394a4e16752 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:18.452630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 982/1963] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:18 2022] POST /identity/v3/users => generated 289 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:18.453633 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fdc25-8d92-4799-ba75-e394a4e16752 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:18.455245 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b47fdc25-8d92-4799-ba75-e394a4e16752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:18.457460 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36320b93-72d7-42cc-88f4-0c0a3ed20485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:18.459569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36320b93-72d7-42cc-88f4-0c0a3ed20485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:18.459863 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36320b93-72d7-42cc-88f4-0c0a3ed20485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:18.460173 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36320b93-72d7-42cc-88f4-0c0a3ed20485 admin admin] PATH_INFO: `/v3/users/1019d1ed18584d8f928230b54450377b` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:18.461470 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36320b93-72d7-42cc-88f4-0c0a3ed20485 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1019d1ed18584d8f928230b54450377b)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:18.463436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36320b93-72d7-42cc-88f4-0c0a3ed20485 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:18.480760 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:18.522189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-36320b93-72d7-42cc-88f4-0c0a3ed20485 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1019d1ed18584d8f928230b54450377b'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:18.531149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fdc25-8d92-4799-ba75-e394a4e16752 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:18.534039 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-36320b93-72d7-42cc-88f4-0c0a3ed20485 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1019d1ed18584d8f928230b54450377b'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:18.535859 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69868a8-b17d-4012-a722-b5a93a43e52b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:18.536220 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69868a8-b17d-4012-a722-b5a93a43e52b service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:18.536583 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69868a8-b17d-4012-a722-b5a93a43e52b service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:18.536934 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69868a8-b17d-4012-a722-b5a93a43e52b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:18.546684 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-36320b93-72d7-42cc-88f4-0c0a3ed20485 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1019d1ed18584d8f928230b54450377b'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:18.547332 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-36320b93-72d7-42cc-88f4-0c0a3ed20485 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1019d1ed18584d8f928230b54450377b'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:18.561900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-36320b93-72d7-42cc-88f4-0c0a3ed20485 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1019d1ed18584d8f928230b54450377b'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:18.569003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 982/1964] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:18 2022] DELETE /identity/v3/users/1019d1ed18584d8f928230b54450377b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:18.569823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36320b93-72d7-42cc-88f4-0c0a3ed20485 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:18.571784 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-36320b93-72d7-42cc-88f4-0c0a3ed20485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:18.610639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69868a8-b17d-4012-a722-b5a93a43e52b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:18.612143 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69868a8-b17d-4012-a722-b5a93a43e52b service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:18.622968 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 983/1965] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:18.623785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69868a8-b17d-4012-a722-b5a93a43e52b service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:18.625100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a69868a8-b17d-4012-a722-b5a93a43e52b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:18.638710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69868a8-b17d-4012-a722-b5a93a43e52b service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:18.643451 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98cf38-bc23-4d5f-9e0c-b48d49ad21ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:18.644339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98cf38-bc23-4d5f-9e0c-b48d49ad21ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:18.644630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98cf38-bc23-4d5f-9e0c-b48d49ad21ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:18.644913 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98cf38-bc23-4d5f-9e0c-b48d49ad21ea admin admin] PATH_INFO: `/v3/projects/35c3fcf9b8634dbd9ce799192830a1e2` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:18.646117 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98cf38-bc23-4d5f-9e0c-b48d49ad21ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=35c3fcf9b8634dbd9ce799192830a1e2)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:18.646671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36320b93-72d7-42cc-88f4-0c0a3ed20485 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:18.648063 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98cf38-bc23-4d5f-9e0c-b48d49ad21ea admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:18.652146 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458cf65-df31-4974-bba5-dc615afa6733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:18.652871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458cf65-df31-4974-bba5-dc615afa6733 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:18.653155 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458cf65-df31-4974-bba5-dc615afa6733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:18.653436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458cf65-df31-4974-bba5-dc615afa6733 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:18.654165 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458cf65-df31-4974-bba5-dc615afa6733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:18.655774 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458cf65-df31-4974-bba5-dc615afa6733 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:18.669126 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 983/1966] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:18.671152 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efa890-1597-4b9e-8a81-5d10de972a8a None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:18.671460 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efa890-1597-4b9e-8a81-5d10de972a8a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:18.671748 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efa890-1597-4b9e-8a81-5d10de972a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:18.693192 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-57efa890-1597-4b9e-8a81-5d10de972a8a None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:18.694580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-57efa890-1597-4b9e-8a81-5d10de972a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:18.707486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 984/1967] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:18.708599 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efa890-1597-4b9e-8a81-5d10de972a8a None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:18.722477 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f98cf38-bc23-4d5f-9e0c-b48d49ad21ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35c3fcf9b8634dbd9ce799192830a1e2', 'request_id': 'req-0f98cf38-bc23-4d5f-9e0c-b48d49ad21ea'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:18.724949 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efa890-1597-4b9e-8a81-5d10de972a8a None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:18.730036 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f98cf38-bc23-4d5f-9e0c-b48d49ad21ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35c3fcf9b8634dbd9ce799192830a1e2', 'request_id': 'req-0f98cf38-bc23-4d5f-9e0c-b48d49ad21ea'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:18.738941 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f98cf38-bc23-4d5f-9e0c-b48d49ad21ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35c3fcf9b8634dbd9ce799192830a1e2', 'request_id': 'req-0f98cf38-bc23-4d5f-9e0c-b48d49ad21ea'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:18.741059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:18.759307 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f98cf38-bc23-4d5f-9e0c-b48d49ad21ea admin admin] The token cache is being invalidate because project 35c3fcf9b8634dbd9ce799192830a1e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:18.759815 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f98cf38-bc23-4d5f-9e0c-b48d49ad21ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:18.761174 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f98cf38-bc23-4d5f-9e0c-b48d49ad21ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:18.766286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 984/1968] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:18 2022] DELETE /identity/v3/projects/35c3fcf9b8634dbd9ce799192830a1e2 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:18.766286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98cf38-bc23-4d5f-9e0c-b48d49ad21ea admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:18.766286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0f98cf38-bc23-4d5f-9e0c-b48d49ad21ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:18.798111 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9a6a6-a715-41a1-9727-41b7459b7fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:18.798807 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9a6a6-a715-41a1-9727-41b7459b7fee admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:18.799109 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9a6a6-a715-41a1-9727-41b7459b7fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:18.799371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9a6a6-a715-41a1-9727-41b7459b7fee admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:18.800149 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff9a6a6-a715-41a1-9727-41b7459b7fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:18.801963 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff9a6a6-a715-41a1-9727-41b7459b7fee admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:18.815148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 985/1969] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:18.816830 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9a6a6-a715-41a1-9727-41b7459b7fee admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:18.821043 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-aff9a6a6-a715-41a1-9727-41b7459b7fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:18.834448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9a6a6-a715-41a1-9727-41b7459b7fee admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:18.839238 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c14318-565a-4983-8f8a-b862b7efbffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:18.840140 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c14318-565a-4983-8f8a-b862b7efbffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:18.840433 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c14318-565a-4983-8f8a-b862b7efbffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:18.840722 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c14318-565a-4983-8f8a-b862b7efbffc admin admin] PATH_INFO: `/v3/users/cfa2bb47a4a54430b57900ec47b8330a` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:18.842462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c14318-565a-4983-8f8a-b862b7efbffc admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfa2bb47a4a54430b57900ec47b8330a)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:18.845170 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c14318-565a-4983-8f8a-b862b7efbffc admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:18.853425 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98cf38-bc23-4d5f-9e0c-b48d49ad21ea admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:18.861906 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40226f47-c12d-4a41-98b7-f81c2b331362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:18.863315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40226f47-c12d-4a41-98b7-f81c2b331362 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:18.863315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40226f47-c12d-4a41-98b7-f81c2b331362 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:18.863516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40226f47-c12d-4a41-98b7-f81c2b331362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:18.865875 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-40226f47-c12d-4a41-98b7-f81c2b331362 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:18.867611 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:18.888518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40226f47-c12d-4a41-98b7-f81c2b331362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:18.890510 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40226f47-c12d-4a41-98b7-f81c2b331362 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:18.904435 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 985/1970] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:18.915341 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-07c14318-565a-4983-8f8a-b862b7efbffc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfa2bb47a4a54430b57900ec47b8330a'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:18.917793 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40226f47-c12d-4a41-98b7-f81c2b331362 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:18.924531 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-40226f47-c12d-4a41-98b7-f81c2b331362 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:18.927340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-07c14318-565a-4983-8f8a-b862b7efbffc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfa2bb47a4a54430b57900ec47b8330a'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:18.937887 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-07c14318-565a-4983-8f8a-b862b7efbffc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfa2bb47a4a54430b57900ec47b8330a'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:18.939045 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-07c14318-565a-4983-8f8a-b862b7efbffc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfa2bb47a4a54430b57900ec47b8330a'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:18.947296 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40226f47-c12d-4a41-98b7-f81c2b331362 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:18.961799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63b7bc-8151-4815-abce-d14872550b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:18.963545 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63b7bc-8151-4815-abce-d14872550b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:18.964318 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63b7bc-8151-4815-abce-d14872550b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:18.964525 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-07c14318-565a-4983-8f8a-b862b7efbffc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfa2bb47a4a54430b57900ec47b8330a'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:18.964654 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63b7bc-8151-4815-abce-d14872550b50 admin admin] PATH_INFO: `/v3/projects/f9708567c672427da37c4dc9f5e02b42/users/a197a1588b94405d90bbb9cb45ed855b/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:18.975014 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 986/1971] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:18 2022] DELETE /identity/v3/users/cfa2bb47a4a54430b57900ec47b8330a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:19.016416 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb63b7bc-8151-4815-abce-d14872550b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9708567c672427da37c4dc9f5e02b42, user_id=a197a1588b94405d90bbb9cb45ed855b, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:19.025788 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb63b7bc-8151-4815-abce-d14872550b50 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:19.065575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 986/1972] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:18 2022] PUT /identity/v3/projects/f9708567c672427da37c4dc9f5e02b42/users/a197a1588b94405d90bbb9cb45ed855b/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:19.078080 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dac4d-cdb1-4ea6-ad0e-8fad2856e73f None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:19.078451 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dac4d-cdb1-4ea6-ad0e-8fad2856e73f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:19.078811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dac4d-cdb1-4ea6-ad0e-8fad2856e73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:19.137392 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-672dac4d-cdb1-4ea6-ad0e-8fad2856e73f None None] MFA Rules not processed for user `a197a1588b94405d90bbb9cb45ed855b`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:19.138778 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-672dac4d-cdb1-4ea6-ad0e-8fad2856e73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:19.266235 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 987/1973] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:19.281897 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63b7bc-8151-4815-abce-d14872550b50 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:19.284075 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fb63b7bc-8151-4815-abce-d14872550b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:19.432131 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63b7bc-8151-4815-abce-d14872550b50 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:19.436586 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63d9d7-f67c-4e64-bf89-7602776782d5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '503a8228d4754c309fc3f365f4cfc61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:19.437415 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63d9d7-f67c-4e64-bf89-7602776782d5 service glance] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:19.437773 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63d9d7-f67c-4e64-bf89-7602776782d5 service glance] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:19.438538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672dac4d-cdb1-4ea6-ad0e-8fad2856e73f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:19.438919 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63d9d7-f67c-4e64-bf89-7602776782d5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:19.440803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-672dac4d-cdb1-4ea6-ad0e-8fad2856e73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:19.472387 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63d9d7-f67c-4e64-bf89-7602776782d5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:19.473928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63d9d7-f67c-4e64-bf89-7602776782d5 service glance] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:19.497654 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 987/1974] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:19.498584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63d9d7-f67c-4e64-bf89-7602776782d5 service glance] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:19.503536 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0c63d9d7-f67c-4e64-bf89-7602776782d5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:19.558946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672dac4d-cdb1-4ea6-ad0e-8fad2856e73f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:19.563415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ebfa3b-3559-4aa8-91dc-4c95c8d70fb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:19.564123 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ebfa3b-3559-4aa8-91dc-4c95c8d70fb2 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:19.564554 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ebfa3b-3559-4aa8-91dc-4c95c8d70fb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:19.564844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ebfa3b-3559-4aa8-91dc-4c95c8d70fb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:19.567116 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-90ebfa3b-3559-4aa8-91dc-4c95c8d70fb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:19.603604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:19.621904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:19.670477 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63d9d7-f67c-4e64-bf89-7602776782d5 service glance] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:19.676379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddb672-0a07-40f4-b81a-3d55caf18337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:19.676717 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ebfa3b-3559-4aa8-91dc-4c95c8d70fb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:19.678160 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddb672-0a07-40f4-b81a-3d55caf18337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:19.678457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddb672-0a07-40f4-b81a-3d55caf18337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:19.678809 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddb672-0a07-40f4-b81a-3d55caf18337 admin admin] PATH_INFO: `/v3/projects/cf0f91c3ad3544b6b2d7e3c5e22eb8c3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:19.678919 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ebfa3b-3559-4aa8-91dc-4c95c8d70fb2 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:19.680740 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddb672-0a07-40f4-b81a-3d55caf18337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf0f91c3ad3544b6b2d7e3c5e22eb8c3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:19.683114 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddb672-0a07-40f4-b81a-3d55caf18337 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:19.705407 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 988/1975] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:19.738510 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ebfa3b-3559-4aa8-91dc-4c95c8d70fb2 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:19.739897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-90ebfa3b-3559-4aa8-91dc-4c95c8d70fb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:19.756428 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b6ddb672-0a07-40f4-b81a-3d55caf18337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf0f91c3ad3544b6b2d7e3c5e22eb8c3', 'request_id': 'req-b6ddb672-0a07-40f4-b81a-3d55caf18337'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:19.764853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ebfa3b-3559-4aa8-91dc-4c95c8d70fb2 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:19.765024 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b6ddb672-0a07-40f4-b81a-3d55caf18337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf0f91c3ad3544b6b2d7e3c5e22eb8c3', 'request_id': 'req-b6ddb672-0a07-40f4-b81a-3d55caf18337'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:19.775353 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b6ddb672-0a07-40f4-b81a-3d55caf18337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf0f91c3ad3544b6b2d7e3c5e22eb8c3', 'request_id': 'req-b6ddb672-0a07-40f4-b81a-3d55caf18337'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:19.789936 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b6ddb672-0a07-40f4-b81a-3d55caf18337 admin admin] The token cache is being invalidate because project cf0f91c3ad3544b6b2d7e3c5e22eb8c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:19.790240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b6ddb672-0a07-40f4-b81a-3d55caf18337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:19.790919 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b6ddb672-0a07-40f4-b81a-3d55caf18337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:19.795992 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 988/1976] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:19 2022] DELETE /identity/v3/projects/cf0f91c3ad3544b6b2d7e3c5e22eb8c3 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:19.800760 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b04b1-4914-412b-aa6b-cc4a98b41749 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:19.801647 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b04b1-4914-412b-aa6b-cc4a98b41749 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:19.803567 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b04b1-4914-412b-aa6b-cc4a98b41749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:19.833311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b97b04b1-4914-412b-aa6b-cc4a98b41749 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:19.835102 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b97b04b1-4914-412b-aa6b-cc4a98b41749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:19.842548 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d773ce-db63-4e89-a2b4-72b60ebaee19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:19.844271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d773ce-db63-4e89-a2b4-72b60ebaee19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:19.844813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d773ce-db63-4e89-a2b4-72b60ebaee19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:19.845238 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d773ce-db63-4e89-a2b4-72b60ebaee19 admin admin] PATH_INFO: `/v3/projects/b828525ebf8c4afe8ad2e36944a6a0fd` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:19.847247 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d773ce-db63-4e89-a2b4-72b60ebaee19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b828525ebf8c4afe8ad2e36944a6a0fd)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:19.850337 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d773ce-db63-4e89-a2b4-72b60ebaee19 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:19.911495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 989/1977] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:09:19.920884 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-64d773ce-db63-4e89-a2b4-72b60ebaee19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b828525ebf8c4afe8ad2e36944a6a0fd', 'request_id': 'req-64d773ce-db63-4e89-a2b4-72b60ebaee19'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:19.922085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4a2b9-2fd5-44b7-b4fa-856bd187dcd2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:19.922826 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4a2b9-2fd5-44b7-b4fa-856bd187dcd2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:19.923401 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4a2b9-2fd5-44b7-b4fa-856bd187dcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:19.945117 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-64d773ce-db63-4e89-a2b4-72b60ebaee19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b828525ebf8c4afe8ad2e36944a6a0fd', 'request_id': 'req-64d773ce-db63-4e89-a2b4-72b60ebaee19'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:19.964903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-42c4a2b9-2fd5-44b7-b4fa-856bd187dcd2 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:19.966453 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-42c4a2b9-2fd5-44b7-b4fa-856bd187dcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:19.974695 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-64d773ce-db63-4e89-a2b4-72b60ebaee19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b828525ebf8c4afe8ad2e36944a6a0fd', 'request_id': 'req-64d773ce-db63-4e89-a2b4-72b60ebaee19'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:19.994214 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-64d773ce-db63-4e89-a2b4-72b60ebaee19 admin admin] The token cache is being invalidate because project b828525ebf8c4afe8ad2e36944a6a0fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:19.994459 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-64d773ce-db63-4e89-a2b4-72b60ebaee19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:19.995747 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-64d773ce-db63-4e89-a2b4-72b60ebaee19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:19.997823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 989/1978] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:19 2022] DELETE /identity/v3/projects/b828525ebf8c4afe8ad2e36944a6a0fd => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:20.002853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d773ce-db63-4e89-a2b4-72b60ebaee19 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:20.004599 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-64d773ce-db63-4e89-a2b4-72b60ebaee19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:20.119135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d773ce-db63-4e89-a2b4-72b60ebaee19 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:20.119925 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 990/1979] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:20.123780 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4a2b9-2fd5-44b7-b4fa-856bd187dcd2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:20.135031 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0fd93a-471f-432c-93db-df83829d3470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:20.138832 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fd93a-471f-432c-93db-df83829d3470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:20.139689 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fd93a-471f-432c-93db-df83829d3470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:20.139689 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fd93a-471f-432c-93db-df83829d3470 admin admin] PATH_INFO: `/v3/users/c69945b9919d49d69888168368d48377` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:20.141208 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0fd93a-471f-432c-93db-df83829d3470 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c69945b9919d49d69888168368d48377)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:20.143268 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4a2b9-2fd5-44b7-b4fa-856bd187dcd2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:20.145233 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0fd93a-471f-432c-93db-df83829d3470 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:20.154704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:20.163221 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9720a4c6-9a69-44bb-9899-2ba29ce3d7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:20.164540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720a4c6-9a69-44bb-9899-2ba29ce3d7bd admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:20.164828 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720a4c6-9a69-44bb-9899-2ba29ce3d7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:20.165142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720a4c6-9a69-44bb-9899-2ba29ce3d7bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:20.166321 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9720a4c6-9a69-44bb-9899-2ba29ce3d7bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:20.166505 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9720a4c6-9a69-44bb-9899-2ba29ce3d7bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:20.168113 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9720a4c6-9a69-44bb-9899-2ba29ce3d7bd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:20.189280 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 991/1980] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:20.194142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9720a4c6-9a69-44bb-9899-2ba29ce3d7bd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:20.225899 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9720a4c6-9a69-44bb-9899-2ba29ce3d7bd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:20.237033 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5dc903-a2bd-43bc-ad56-cd741fc0c84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:20.238973 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dc903-a2bd-43bc-ad56-cd741fc0c84a admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:20.239611 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dc903-a2bd-43bc-ad56-cd741fc0c84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:20.240004 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dc903-a2bd-43bc-ad56-cd741fc0c84a admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:20.240344 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-dd0fd93a-471f-432c-93db-df83829d3470 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c69945b9919d49d69888168368d48377'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:20.240984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5dc903-a2bd-43bc-ad56-cd741fc0c84a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:20.245668 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5dc903-a2bd-43bc-ad56-cd741fc0c84a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:20.253515 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-dd0fd93a-471f-432c-93db-df83829d3470 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c69945b9919d49d69888168368d48377'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:20.269526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-dd0fd93a-471f-432c-93db-df83829d3470 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c69945b9919d49d69888168368d48377'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:20.272264 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-dd0fd93a-471f-432c-93db-df83829d3470 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c69945b9919d49d69888168368d48377'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:20.281479 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:20.319692 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-dd0fd93a-471f-432c-93db-df83829d3470 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c69945b9919d49d69888168368d48377'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:20.337558 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 990/1981] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:20 2022] DELETE /identity/v3/users/c69945b9919d49d69888168368d48377 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:20.391290 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 992/1982] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:20 2022] POST /identity/v3/projects => generated 361 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:20.396968 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0fd93a-471f-432c-93db-df83829d3470 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:20.401522 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-dd0fd93a-471f-432c-93db-df83829d3470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:20.550499 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0fd93a-471f-432c-93db-df83829d3470 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:20.627452 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2a9ab-aacf-422b-9768-4d9f672d56bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:20.628939 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2a9ab-aacf-422b-9768-4d9f672d56bc admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:20.629580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2a9ab-aacf-422b-9768-4d9f672d56bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:20.629878 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2a9ab-aacf-422b-9768-4d9f672d56bc admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:20.630613 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f2a9ab-aacf-422b-9768-4d9f672d56bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:20.632768 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f2a9ab-aacf-422b-9768-4d9f672d56bc admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:20.662812 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 991/1983] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:20 2022] POST /identity/v3/users => generated 281 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:20.663571 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5dc903-a2bd-43bc-ad56-cd741fc0c84a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:20.677348 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5dc903-a2bd-43bc-ad56-cd741fc0c84a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:20.700372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca57566-2623-467d-b943-1fd75e35a9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:20.701764 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca57566-2623-467d-b943-1fd75e35a9d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:20.702201 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca57566-2623-467d-b943-1fd75e35a9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:20.703237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca57566-2623-467d-b943-1fd75e35a9d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:20.708193 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca57566-2623-467d-b943-1fd75e35a9d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:20.712732 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca57566-2623-467d-b943-1fd75e35a9d2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:20.739477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2a9ab-aacf-422b-9768-4d9f672d56bc admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:20.741498 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e9f2a9ab-aacf-422b-9768-4d9f672d56bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:20.741810 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 993/1984] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:09:20.746350 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca57566-2623-467d-b943-1fd75e35a9d2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:20.759909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2a9ab-aacf-422b-9768-4d9f672d56bc admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:20.770749 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca57566-2623-467d-b943-1fd75e35a9d2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:20.774428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bca02f-a7db-4b1a-a8dd-9788f42a14eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:20.776668 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bca02f-a7db-4b1a-a8dd-9788f42a14eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:20.777456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bca02f-a7db-4b1a-a8dd-9788f42a14eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:20.778191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bca02f-a7db-4b1a-a8dd-9788f42a14eb admin admin] PATH_INFO: `/v3/projects/c2aaa5ed1bd6459ba22188a1b7e34b5c` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:20.780279 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf2912-9c2d-44cc-b5ce-4b50086fcdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:20.781088 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf2912-9c2d-44cc-b5ce-4b50086fcdfb admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:20.781446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf2912-9c2d-44cc-b5ce-4b50086fcdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:20.781837 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf2912-9c2d-44cc-b5ce-4b50086fcdfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:20.782871 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf2912-9c2d-44cc-b5ce-4b50086fcdfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:20.784796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bca02f-a7db-4b1a-a8dd-9788f42a14eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2aaa5ed1bd6459ba22188a1b7e34b5c)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:20.786948 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf2912-9c2d-44cc-b5ce-4b50086fcdfb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:20.789904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bca02f-a7db-4b1a-a8dd-9788f42a14eb admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:20.814136 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 994/1985] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:09:20.818429 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf2912-9c2d-44cc-b5ce-4b50086fcdfb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:20.833443 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:20.839325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf2912-9c2d-44cc-b5ce-4b50086fcdfb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:20.868077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82614e2b-ead5-4e49-88b1-3e31fbc76cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:20.869020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82614e2b-ead5-4e49-88b1-3e31fbc76cf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:20.869331 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82614e2b-ead5-4e49-88b1-3e31fbc76cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:20.869565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82614e2b-ead5-4e49-88b1-3e31fbc76cf0 admin admin] PATH_INFO: `/v3/projects/72df6c75d2b040c289a507d90f601fad/users/67c8d963e2c443b79210070b051c10e6/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:20.927701 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-09bca02f-a7db-4b1a-a8dd-9788f42a14eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2aaa5ed1bd6459ba22188a1b7e34b5c', 'request_id': 'req-09bca02f-a7db-4b1a-a8dd-9788f42a14eb'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:20.929395 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82614e2b-ead5-4e49-88b1-3e31fbc76cf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72df6c75d2b040c289a507d90f601fad, user_id=67c8d963e2c443b79210070b051c10e6, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:20.933647 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82614e2b-ead5-4e49-88b1-3e31fbc76cf0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:20.952450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-09bca02f-a7db-4b1a-a8dd-9788f42a14eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2aaa5ed1bd6459ba22188a1b7e34b5c', 'request_id': 'req-09bca02f-a7db-4b1a-a8dd-9788f42a14eb'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:20.972118 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-09bca02f-a7db-4b1a-a8dd-9788f42a14eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2aaa5ed1bd6459ba22188a1b7e34b5c', 'request_id': 'req-09bca02f-a7db-4b1a-a8dd-9788f42a14eb'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:20.984533 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 995/1986] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:20 2022] PUT /identity/v3/projects/72df6c75d2b040c289a507d90f601fad/users/67c8d963e2c443b79210070b051c10e6/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:20.994140 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-09bca02f-a7db-4b1a-a8dd-9788f42a14eb admin admin] The token cache is being invalidate because project c2aaa5ed1bd6459ba22188a1b7e34b5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:20.994140 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-09bca02f-a7db-4b1a-a8dd-9788f42a14eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:20.994140 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-09bca02f-a7db-4b1a-a8dd-9788f42a14eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:20.995296 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 992/1987] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:20 2022] DELETE /identity/v3/projects/c2aaa5ed1bd6459ba22188a1b7e34b5c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:20.996638 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dfe34-20e7-48a2-99d6-3dd631d5b717 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:20.996938 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dfe34-20e7-48a2-99d6-3dd631d5b717 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:20.997218 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dfe34-20e7-48a2-99d6-3dd631d5b717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:21.000338 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a81d4-eea4-46fd-9a57-21875641a201 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:21.000627 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a81d4-eea4-46fd-9a57-21875641a201 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:21.000915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a81d4-eea4-46fd-9a57-21875641a201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:21.028070 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-7b9a81d4-eea4-46fd-9a57-21875641a201 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:21.029300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7b9a81d4-eea4-46fd-9a57-21875641a201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:21.047491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-993dfe34-20e7-48a2-99d6-3dd631d5b717 None None] MFA Rules not processed for user `67c8d963e2c443b79210070b051c10e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:21.049966 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-993dfe34-20e7-48a2-99d6-3dd631d5b717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:21.129718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 993/1988] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:09:21.131770 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951c691-1668-4aae-9e58-95f59372edcf None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:21.132274 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951c691-1668-4aae-9e58-95f59372edcf None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:21.133271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951c691-1668-4aae-9e58-95f59372edcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:21.159987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-6951c691-1668-4aae-9e58-95f59372edcf None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:21.172494 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6951c691-1668-4aae-9e58-95f59372edcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:21.179221 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 996/1989] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:20 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:09:21.182284 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 994/1990] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:09:21.186295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993dfe34-20e7-48a2-99d6-3dd631d5b717 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:21.197936 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993dfe34-20e7-48a2-99d6-3dd631d5b717 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:21.202633 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb884f1-b7e2-405c-b934-515bf8e76a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:21.203292 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb884f1-b7e2-405c-b934-515bf8e76a39 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:21.203635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb884f1-b7e2-405c-b934-515bf8e76a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:21.203844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb884f1-b7e2-405c-b934-515bf8e76a39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:21.204659 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb884f1-b7e2-405c-b934-515bf8e76a39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:21.204971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb884f1-b7e2-405c-b934-515bf8e76a39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:21.206645 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb884f1-b7e2-405c-b934-515bf8e76a39 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:21.222458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 997/1991] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:21.227281 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951c691-1668-4aae-9e58-95f59372edcf None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:21.245000 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951c691-1668-4aae-9e58-95f59372edcf None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:21.252401 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d9972-e063-4734-a18c-e9605ca4a3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:21.253406 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d9972-e063-4734-a18c-e9605ca4a3ee admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:21.253793 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d9972-e063-4734-a18c-e9605ca4a3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:21.254299 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d9972-e063-4734-a18c-e9605ca4a3ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:21.255028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d9972-e063-4734-a18c-e9605ca4a3ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:21.258713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb884f1-b7e2-405c-b934-515bf8e76a39 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:21.259268 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d9972-e063-4734-a18c-e9605ca4a3ee admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:21.260979 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5cb884f1-b7e2-405c-b934-515bf8e76a39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:21.292780 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 995/1992] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:21 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:21.299795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d9972-e063-4734-a18c-e9605ca4a3ee admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:21.354339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb884f1-b7e2-405c-b934-515bf8e76a39 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:21.409756 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:21.419698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d9972-e063-4734-a18c-e9605ca4a3ee admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:21.430366 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95c539-25a8-409b-b7a1-9b23f0f0b025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:21.431176 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95c539-25a8-409b-b7a1-9b23f0f0b025 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:21.431547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95c539-25a8-409b-b7a1-9b23f0f0b025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:21.431832 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95c539-25a8-409b-b7a1-9b23f0f0b025 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:21.432568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95c539-25a8-409b-b7a1-9b23f0f0b025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:21.434511 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95c539-25a8-409b-b7a1-9b23f0f0b025 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:21.436352 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ddc8f8-8aa4-4108-ba35-a9355bf8cb23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:21.437251 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ddc8f8-8aa4-4108-ba35-a9355bf8cb23 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:21.437500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ddc8f8-8aa4-4108-ba35-a9355bf8cb23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:21.437822 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ddc8f8-8aa4-4108-ba35-a9355bf8cb23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:21.457039 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 996/1993] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:21 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:09:21.461473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95c539-25a8-409b-b7a1-9b23f0f0b025 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:21.472914 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95c539-25a8-409b-b7a1-9b23f0f0b025 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:21.481386 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b26e6-1531-49bf-af06-2620c9272cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:21.481618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b26e6-1531-49bf-af06-2620c9272cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:21.481917 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b26e6-1531-49bf-af06-2620c9272cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:21.482205 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b26e6-1531-49bf-af06-2620c9272cc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:21.483087 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b26e6-1531-49bf-af06-2620c9272cc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:21.485111 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b26e6-1531-49bf-af06-2620c9272cc4 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:21.486717 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:21.503800 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 997/1994] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:09:21.504819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b26e6-1531-49bf-af06-2620c9272cc4 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:21.509535 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ddc8f8-8aa4-4108-ba35-a9355bf8cb23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:21.510998 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ddc8f8-8aa4-4108-ba35-a9355bf8cb23 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:21.515182 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b26e6-1531-49bf-af06-2620c9272cc4 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:21.520105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 998/1995] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:21 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:21.520223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93baa56-b41b-4bdd-9cfe-4a2501ef867d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:21.521062 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93baa56-b41b-4bdd-9cfe-4a2501ef867d admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:21.521400 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93baa56-b41b-4bdd-9cfe-4a2501ef867d admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:21.521689 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93baa56-b41b-4bdd-9cfe-4a2501ef867d admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:21.522414 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93baa56-b41b-4bdd-9cfe-4a2501ef867d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:21.524774 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93baa56-b41b-4bdd-9cfe-4a2501ef867d admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:21.534798 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ddc8f8-8aa4-4108-ba35-a9355bf8cb23 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:21.535831 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 998/1996] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:21.536202 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-91ddc8f8-8aa4-4108-ba35-a9355bf8cb23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:21.539491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93baa56-b41b-4bdd-9cfe-4a2501ef867d admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:21.550381 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ddc8f8-8aa4-4108-ba35-a9355bf8cb23 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:21.550730 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93baa56-b41b-4bdd-9cfe-4a2501ef867d admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:21.554969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009f5d8-9a94-4494-af26-885d968cf133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:21.555350 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa6b9e-76d1-43a3-9277-569879fea1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:21.555842 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009f5d8-9a94-4494-af26-885d968cf133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:21.556163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009f5d8-9a94-4494-af26-885d968cf133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:21.556271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa6b9e-76d1-43a3-9277-569879fea1b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:21.556448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009f5d8-9a94-4494-af26-885d968cf133 admin admin] PATH_INFO: `/v3/users/67c8d963e2c443b79210070b051c10e6` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:21.556645 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa6b9e-76d1-43a3-9277-569879fea1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:21.556794 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa6b9e-76d1-43a3-9277-569879fea1b4 admin admin] PATH_INFO: `/v3/projects/ba65aa2594a24b669af81620d2af7ef5/users/2c6a5983bee24f438becc13b28bd0405/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:21.557649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9009f5d8-9a94-4494-af26-885d968cf133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67c8d963e2c443b79210070b051c10e6)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:21.559573 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9009f5d8-9a94-4494-af26-885d968cf133 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:21.572680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa6b9e-76d1-43a3-9277-569879fea1b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba65aa2594a24b669af81620d2af7ef5, user_id=2c6a5983bee24f438becc13b28bd0405, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:21.575710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa6b9e-76d1-43a3-9277-569879fea1b4 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:21.589343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 999/1997] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:21 2022] PUT /identity/v3/projects/ba65aa2594a24b669af81620d2af7ef5/users/2c6a5983bee24f438becc13b28bd0405/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:21.599979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd3aed-bbc2-4242-b9c9-684a7135ddff None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:21.600288 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd3aed-bbc2-4242-b9c9-684a7135ddff None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:21.600574 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd3aed-bbc2-4242-b9c9-684a7135ddff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:21.611480 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9009f5d8-9a94-4494-af26-885d968cf133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67c8d963e2c443b79210070b051c10e6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:21.620274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9009f5d8-9a94-4494-af26-885d968cf133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67c8d963e2c443b79210070b051c10e6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:21.627275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9009f5d8-9a94-4494-af26-885d968cf133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67c8d963e2c443b79210070b051c10e6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:21.627643 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f5fd3aed-bbc2-4242-b9c9-684a7135ddff None None] MFA Rules not processed for user `2c6a5983bee24f438becc13b28bd0405`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:21.628068 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9009f5d8-9a94-4494-af26-885d968cf133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67c8d963e2c443b79210070b051c10e6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:21.628931 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f5fd3aed-bbc2-4242-b9c9-684a7135ddff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:21.642347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9009f5d8-9a94-4494-af26-885d968cf133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67c8d963e2c443b79210070b051c10e6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:21.649024 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 999/1998] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:21 2022] DELETE /identity/v3/users/67c8d963e2c443b79210070b051c10e6 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:21.653491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271181a-936d-48d3-be46-f4d3e95f3339 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:21.653783 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271181a-936d-48d3-be46-f4d3e95f3339 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:21.654076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271181a-936d-48d3-be46-f4d3e95f3339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:21.671291 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6271181a-936d-48d3-be46-f4d3e95f3339 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:21.672458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6271181a-936d-48d3-be46-f4d3e95f3339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:21.708627 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1000/1999] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:09:21.714309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd3aed-bbc2-4242-b9c9-684a7135ddff None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:21.715962 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f5fd3aed-bbc2-4242-b9c9-684a7135ddff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:21.734149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:21.750396 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1000/2000] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:21.760461 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271181a-936d-48d3-be46-f4d3e95f3339 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:21.761926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6271181a-936d-48d3-be46-f4d3e95f3339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:21.797763 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd3aed-bbc2-4242-b9c9-684a7135ddff None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:21.803885 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9b4b1-2544-45ab-8414-7aa5254c4149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:21.804494 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9b4b1-2544-45ab-8414-7aa5254c4149 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:21.808237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9b4b1-2544-45ab-8414-7aa5254c4149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:21.808621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9b4b1-2544-45ab-8414-7aa5254c4149 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:21.809421 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9b4b1-2544-45ab-8414-7aa5254c4149 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:21.812714 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9b4b1-2544-45ab-8414-7aa5254c4149 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:21.833386 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6271181a-936d-48d3-be46-f4d3e95f3339 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:21.839465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046c5e8-e1f5-4d0a-8a21-0ff522e20742 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:21.840909 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046c5e8-e1f5-4d0a-8a21-0ff522e20742 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:21.841364 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046c5e8-e1f5-4d0a-8a21-0ff522e20742 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:21.841751 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046c5e8-e1f5-4d0a-8a21-0ff522e20742 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:21.843852 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1001/2001] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:21 2022] POST /identity/v3/projects => generated 375 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:21.847882 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9b4b1-2544-45ab-8414-7aa5254c4149 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:21.860746 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046c5e8-e1f5-4d0a-8a21-0ff522e20742 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:21.862181 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046c5e8-e1f5-4d0a-8a21-0ff522e20742 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:21.872120 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1001/2002] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:09:21.894505 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:21.929953 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9b4b1-2544-45ab-8414-7aa5254c4149 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:21.934922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80a239-ead7-48f8-aad6-95353a5a0af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:21.935711 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80a239-ead7-48f8-aad6-95353a5a0af6 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:21.936039 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80a239-ead7-48f8-aad6-95353a5a0af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:21.936332 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80a239-ead7-48f8-aad6-95353a5a0af6 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:21.937294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80a239-ead7-48f8-aad6-95353a5a0af6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:21.939265 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80a239-ead7-48f8-aad6-95353a5a0af6 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:21.961511 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1002/2003] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:21 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:09:21.965360 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046c5e8-e1f5-4d0a-8a21-0ff522e20742 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:21.976952 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046c5e8-e1f5-4d0a-8a21-0ff522e20742 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:21.981802 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af0094-27b7-4046-a323-ff33223ca2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:21.982581 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af0094-27b7-4046-a323-ff33223ca2bc admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:21.982873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af0094-27b7-4046-a323-ff33223ca2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:21.983158 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af0094-27b7-4046-a323-ff33223ca2bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:21.983891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af0094-27b7-4046-a323-ff33223ca2bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:21.985527 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af0094-27b7-4046-a323-ff33223ca2bc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:21.998615 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1002/2004] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:22.000174 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80a239-ead7-48f8-aad6-95353a5a0af6 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:22.011724 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80a239-ead7-48f8-aad6-95353a5a0af6 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:22.016784 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c9951-edd2-43b8-a897-2572b25fe4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:22.017468 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c9951-edd2-43b8-a897-2572b25fe4fb admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:22.017758 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c9951-edd2-43b8-a897-2572b25fe4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:22.018038 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c9951-edd2-43b8-a897-2572b25fe4fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:22.019129 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c9951-edd2-43b8-a897-2572b25fe4fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:22.021613 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c9951-edd2-43b8-a897-2572b25fe4fb admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:22.036883 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1003/2005] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:22.037754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af0094-27b7-4046-a323-ff33223ca2bc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:22.048736 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af0094-27b7-4046-a323-ff33223ca2bc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:22.053940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba405808-912f-4c15-9040-a6860974a59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:22.054832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba405808-912f-4c15-9040-a6860974a59d admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:22.055121 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba405808-912f-4c15-9040-a6860974a59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:22.055402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba405808-912f-4c15-9040-a6860974a59d admin admin] PATH_INFO: `/v3/projects/d9e5ffa48c7d4b1188d1daf826efcfaf/users/49bfbbc1612e467b808e7a9acddab665/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:22.071366 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba405808-912f-4c15-9040-a6860974a59d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9e5ffa48c7d4b1188d1daf826efcfaf, user_id=49bfbbc1612e467b808e7a9acddab665, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:22.074162 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba405808-912f-4c15-9040-a6860974a59d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:22.089224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1003/2006] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:22 2022] PUT /identity/v3/projects/d9e5ffa48c7d4b1188d1daf826efcfaf/users/49bfbbc1612e467b808e7a9acddab665/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:22.101182 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cda05c-88e8-48a9-8c46-bf717c1bdbd2 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:22.102128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cda05c-88e8-48a9-8c46-bf717c1bdbd2 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:22.102407 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cda05c-88e8-48a9-8c46-bf717c1bdbd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:22.134119 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-50cda05c-88e8-48a9-8c46-bf717c1bdbd2 None None] MFA Rules not processed for user `49bfbbc1612e467b808e7a9acddab665`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:22.135576 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-50cda05c-88e8-48a9-8c46-bf717c1bdbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:22.169670 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba405808-912f-4c15-9040-a6860974a59d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:22.171623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ba405808-912f-4c15-9040-a6860974a59d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:22.240501 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1004/2007] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:22.246216 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cda05c-88e8-48a9-8c46-bf717c1bdbd2 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:22.257404 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba405808-912f-4c15-9040-a6860974a59d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:22.259890 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cda05c-88e8-48a9-8c46-bf717c1bdbd2 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:22.263535 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483e999-b683-4a9b-92e6-1622b601d80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:22.264656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483e999-b683-4a9b-92e6-1622b601d80f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:22.264959 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483e999-b683-4a9b-92e6-1622b601d80f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:22.265244 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483e999-b683-4a9b-92e6-1622b601d80f admin admin] PATH_INFO: `/v3/projects/72df6c75d2b040c289a507d90f601fad` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:22.266485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f483e999-b683-4a9b-92e6-1622b601d80f admin admin] RBAC: Authorizing `identity:delete_project(project_id=72df6c75d2b040c289a507d90f601fad)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:22.266790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24681f-8a3f-4b3f-a1f6-d54db9dcc20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:22.267541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24681f-8a3f-4b3f-a1f6-d54db9dcc20c admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:22.267901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24681f-8a3f-4b3f-a1f6-d54db9dcc20c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:22.268549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24681f-8a3f-4b3f-a1f6-d54db9dcc20c admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:22.268655 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f483e999-b683-4a9b-92e6-1622b601d80f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:22.269096 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a24681f-8a3f-4b3f-a1f6-d54db9dcc20c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:22.271820 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a24681f-8a3f-4b3f-a1f6-d54db9dcc20c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:22.307481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1005/2008] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:22 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:09:22.316318 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24681f-8a3f-4b3f-a1f6-d54db9dcc20c admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:22.330657 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f483e999-b683-4a9b-92e6-1622b601d80f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72df6c75d2b040c289a507d90f601fad', 'request_id': 'req-f483e999-b683-4a9b-92e6-1622b601d80f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:22.337836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f483e999-b683-4a9b-92e6-1622b601d80f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72df6c75d2b040c289a507d90f601fad', 'request_id': 'req-f483e999-b683-4a9b-92e6-1622b601d80f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:22.349375 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f483e999-b683-4a9b-92e6-1622b601d80f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72df6c75d2b040c289a507d90f601fad', 'request_id': 'req-f483e999-b683-4a9b-92e6-1622b601d80f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:22.359675 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:22.371930 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f483e999-b683-4a9b-92e6-1622b601d80f admin admin] The token cache is being invalidate because project 72df6c75d2b040c289a507d90f601fad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:22.372180 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f483e999-b683-4a9b-92e6-1622b601d80f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:22.372949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f483e999-b683-4a9b-92e6-1622b601d80f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:22.374987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1004/2009] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:22 2022] DELETE /identity/v3/projects/72df6c75d2b040c289a507d90f601fad => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:22.430448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24681f-8a3f-4b3f-a1f6-d54db9dcc20c admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:22.466950 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:22.518806 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6a681-bb4a-4312-a873-e0f667f676fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:22.519693 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6a681-bb4a-4312-a873-e0f667f676fc admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:22.520045 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6a681-bb4a-4312-a873-e0f667f676fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:22.520294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6a681-bb4a-4312-a873-e0f667f676fc admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:22.521017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6a681-bb4a-4312-a873-e0f667f676fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:22.523052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6a681-bb4a-4312-a873-e0f667f676fc admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:22.558300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1006/2010] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:22 2022] POST /identity/v3/users => generated 294 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:22.565099 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483e999-b683-4a9b-92e6-1622b601d80f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:22.567211 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f483e999-b683-4a9b-92e6-1622b601d80f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:22.583446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483e999-b683-4a9b-92e6-1622b601d80f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:22.588734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3427a0a-1521-4617-b863-68cf0ee982c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:22.589447 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3427a0a-1521-4617-b863-68cf0ee982c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:22.589742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3427a0a-1521-4617-b863-68cf0ee982c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:22.590030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3427a0a-1521-4617-b863-68cf0ee982c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:22.590797 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3427a0a-1521-4617-b863-68cf0ee982c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:22.592652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3427a0a-1521-4617-b863-68cf0ee982c3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:22.604280 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1005/2011] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:22.608323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6a681-bb4a-4312-a873-e0f667f676fc admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:22.625943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6a681-bb4a-4312-a873-e0f667f676fc admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:22.631264 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d96d6-8cb4-41ed-aee5-7f90e5a75e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:22.632224 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d96d6-8cb4-41ed-aee5-7f90e5a75e29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:22.632548 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d96d6-8cb4-41ed-aee5-7f90e5a75e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:22.632840 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d96d6-8cb4-41ed-aee5-7f90e5a75e29 admin admin] PATH_INFO: `/v3/projects/9445c0f76e3c42d5b146da4c5d63287a/users/3f1f3423f24f45f48398900c1639af36/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:22.649435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d96d6-8cb4-41ed-aee5-7f90e5a75e29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9445c0f76e3c42d5b146da4c5d63287a, user_id=3f1f3423f24f45f48398900c1639af36, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:22.652153 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d96d6-8cb4-41ed-aee5-7f90e5a75e29 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:22.667706 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1007/2012] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:22 2022] PUT /identity/v3/projects/9445c0f76e3c42d5b146da4c5d63287a/users/3f1f3423f24f45f48398900c1639af36/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:22.671867 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3427a0a-1521-4617-b863-68cf0ee982c3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:22.682685 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3427a0a-1521-4617-b863-68cf0ee982c3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:22.749444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3548ae6f-0ce0-4216-b2e9-2fc5dc62dda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:22.750127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548ae6f-0ce0-4216-b2e9-2fc5dc62dda9 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:22.750435 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548ae6f-0ce0-4216-b2e9-2fc5dc62dda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:22.750734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548ae6f-0ce0-4216-b2e9-2fc5dc62dda9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:22.751431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3548ae6f-0ce0-4216-b2e9-2fc5dc62dda9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:22.753092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3548ae6f-0ce0-4216-b2e9-2fc5dc62dda9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:22.769841 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1006/2013] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:22.776088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d96d6-8cb4-41ed-aee5-7f90e5a75e29 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:22.786638 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d96d6-8cb4-41ed-aee5-7f90e5a75e29 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:22.791247 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0006f-e675-4131-96fa-dd9b16c42e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:22.792161 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0006f-e675-4131-96fa-dd9b16c42e56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:22.792458 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0006f-e675-4131-96fa-dd9b16c42e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:22.792745 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0006f-e675-4131-96fa-dd9b16c42e56 admin admin] PATH_INFO: `/v3/projects/9445c0f76e3c42d5b146da4c5d63287a/users/3f1f3423f24f45f48398900c1639af36/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:22.794778 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0006f-e675-4131-96fa-dd9b16c42e56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9445c0f76e3c42d5b146da4c5d63287a, user_id=3f1f3423f24f45f48398900c1639af36, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:22.797494 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0006f-e675-4131-96fa-dd9b16c42e56 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:22.817335 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1008/2014] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:22 2022] PUT /identity/v3/projects/9445c0f76e3c42d5b146da4c5d63287a/users/3f1f3423f24f45f48398900c1639af36/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:22.828597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c40ca-8b6e-4e7f-a03c-66e17b1abdcb None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:22.828970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c40ca-8b6e-4e7f-a03c-66e17b1abdcb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:22.829426 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c40ca-8b6e-4e7f-a03c-66e17b1abdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:22.864179 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-973c40ca-8b6e-4e7f-a03c-66e17b1abdcb None None] MFA Rules not processed for user `3f1f3423f24f45f48398900c1639af36`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:22.865436 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-973c40ca-8b6e-4e7f-a03c-66e17b1abdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:22.979007 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1007/2015] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:22 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:22.987752 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0006f-e675-4131-96fa-dd9b16c42e56 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:22.990596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9ee0006f-e675-4131-96fa-dd9b16c42e56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:23.062197 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0006f-e675-4131-96fa-dd9b16c42e56 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:23.068085 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc76e8-e72f-481e-bfde-ade3da69009b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:23.068759 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc76e8-e72f-481e-bfde-ade3da69009b service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:23.069055 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc76e8-e72f-481e-bfde-ade3da69009b service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:23.069338 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc76e8-e72f-481e-bfde-ade3da69009b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:23.071740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fdbc76e8-e72f-481e-bfde-ade3da69009b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:23.136089 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc76e8-e72f-481e-bfde-ade3da69009b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:23.137705 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc76e8-e72f-481e-bfde-ade3da69009b service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:23.147293 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1009/2016] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:24.079481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c40ca-8b6e-4e7f-a03c-66e17b1abdcb None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:24.081190 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-973c40ca-8b6e-4e7f-a03c-66e17b1abdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:24.095149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c40ca-8b6e-4e7f-a03c-66e17b1abdcb None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:24.098802 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949da8d-3c99-4969-901d-b611094b2e63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:24.099455 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949da8d-3c99-4969-901d-b611094b2e63 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:24.099747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949da8d-3c99-4969-901d-b611094b2e63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:24.100073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949da8d-3c99-4969-901d-b611094b2e63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:24.102460 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6949da8d-3c99-4969-901d-b611094b2e63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:24.170275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949da8d-3c99-4969-901d-b611094b2e63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:24.171848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949da8d-3c99-4969-901d-b611094b2e63 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:24.181204 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1008/2017] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:24.735468 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc76e8-e72f-481e-bfde-ade3da69009b service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:24.745212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc76e8-e72f-481e-bfde-ade3da69009b service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:24.749298 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b77df-70ac-43f0-9718-b80e8011c213 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:24.749948 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b77df-70ac-43f0-9718-b80e8011c213 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:24.750279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b77df-70ac-43f0-9718-b80e8011c213 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:24.750563 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b77df-70ac-43f0-9718-b80e8011c213 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:24.752452 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-986b77df-70ac-43f0-9718-b80e8011c213 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:24.787877 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:24.822080 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b77df-70ac-43f0-9718-b80e8011c213 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:24.823535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b77df-70ac-43f0-9718-b80e8011c213 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:24.832888 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1010/2018] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:25.578221 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949da8d-3c99-4969-901d-b611094b2e63 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:25.587205 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949da8d-3c99-4969-901d-b611094b2e63 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:25.591124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705c6d1-4c08-4889-998d-9e4da8f560a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:25.591789 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705c6d1-4c08-4889-998d-9e4da8f560a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:25.592119 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705c6d1-4c08-4889-998d-9e4da8f560a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:25.592401 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705c6d1-4c08-4889-998d-9e4da8f560a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:25.598715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:25.603702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705c6d1-4c08-4889-998d-9e4da8f560a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:25.605198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705c6d1-4c08-4889-998d-9e4da8f560a3 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:25.614617 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1009/2019] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:25 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:26.395401 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b77df-70ac-43f0-9718-b80e8011c213 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:26.396927 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-986b77df-70ac-43f0-9718-b80e8011c213 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:26.463685 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b77df-70ac-43f0-9718-b80e8011c213 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:26.468220 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966ee47-e39e-4e01-bf4b-cde23888823c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:26.469104 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966ee47-e39e-4e01-bf4b-cde23888823c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:26.469395 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966ee47-e39e-4e01-bf4b-cde23888823c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:26.469681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966ee47-e39e-4e01-bf4b-cde23888823c admin admin] PATH_INFO: `/v3/users/e0936c8395d843db8392ca0ef7920e6f` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:26.470874 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966ee47-e39e-4e01-bf4b-cde23888823c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0936c8395d843db8392ca0ef7920e6f)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:26.472885 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966ee47-e39e-4e01-bf4b-cde23888823c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:26.524069 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7966ee47-e39e-4e01-bf4b-cde23888823c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0936c8395d843db8392ca0ef7920e6f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:26.531861 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7966ee47-e39e-4e01-bf4b-cde23888823c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0936c8395d843db8392ca0ef7920e6f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:26.539381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7966ee47-e39e-4e01-bf4b-cde23888823c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0936c8395d843db8392ca0ef7920e6f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:26.540070 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7966ee47-e39e-4e01-bf4b-cde23888823c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0936c8395d843db8392ca0ef7920e6f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:26.552604 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7966ee47-e39e-4e01-bf4b-cde23888823c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0936c8395d843db8392ca0ef7920e6f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:26.559666 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1011/2020] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:26 2022] DELETE /identity/v3/users/e0936c8395d843db8392ca0ef7920e6f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:26.564287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac386e8-6500-46c2-a4ce-f7b935239072 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:26.564711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac386e8-6500-46c2-a4ce-f7b935239072 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:26.565066 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac386e8-6500-46c2-a4ce-f7b935239072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:26.584514 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8ac386e8-6500-46c2-a4ce-f7b935239072 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:26.585827 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8ac386e8-6500-46c2-a4ce-f7b935239072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:26.646695 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1010/2021] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:26.656376 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966ee47-e39e-4e01-bf4b-cde23888823c admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:26.658152 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7966ee47-e39e-4e01-bf4b-cde23888823c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:26.716891 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966ee47-e39e-4e01-bf4b-cde23888823c admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:26.720546 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f5214-ca7a-421e-80dd-bc60f60aa204 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:26.721198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f5214-ca7a-421e-80dd-bc60f60aa204 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:26.721489 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f5214-ca7a-421e-80dd-bc60f60aa204 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:26.721772 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f5214-ca7a-421e-80dd-bc60f60aa204 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:26.738725 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:26.741315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f5214-ca7a-421e-80dd-bc60f60aa204 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:26.744285 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f5214-ca7a-421e-80dd-bc60f60aa204 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:26.756470 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1012/2022] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:27.096225 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac386e8-6500-46c2-a4ce-f7b935239072 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:27.098087 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8ac386e8-6500-46c2-a4ce-f7b935239072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:27.110537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac386e8-6500-46c2-a4ce-f7b935239072 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:27.115604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ade247-ed3a-4156-80db-58fa4198cbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:27.116762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ade247-ed3a-4156-80db-58fa4198cbba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:27.116913 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ade247-ed3a-4156-80db-58fa4198cbba admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:27.117272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ade247-ed3a-4156-80db-58fa4198cbba admin admin] PATH_INFO: `/v3/projects/42e0babfb4c14e3a834da6b8fa45b9e9` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:27.118321 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ade247-ed3a-4156-80db-58fa4198cbba admin admin] RBAC: Authorizing `identity:delete_project(project_id=42e0babfb4c14e3a834da6b8fa45b9e9)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:27.120348 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ade247-ed3a-4156-80db-58fa4198cbba admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:27.175234 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c4ade247-ed3a-4156-80db-58fa4198cbba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42e0babfb4c14e3a834da6b8fa45b9e9', 'request_id': 'req-c4ade247-ed3a-4156-80db-58fa4198cbba'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:27.183716 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c4ade247-ed3a-4156-80db-58fa4198cbba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42e0babfb4c14e3a834da6b8fa45b9e9', 'request_id': 'req-c4ade247-ed3a-4156-80db-58fa4198cbba'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:27.194589 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c4ade247-ed3a-4156-80db-58fa4198cbba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42e0babfb4c14e3a834da6b8fa45b9e9', 'request_id': 'req-c4ade247-ed3a-4156-80db-58fa4198cbba'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:27.208249 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c4ade247-ed3a-4156-80db-58fa4198cbba admin admin] The token cache is being invalidate because project 42e0babfb4c14e3a834da6b8fa45b9e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:27.208634 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c4ade247-ed3a-4156-80db-58fa4198cbba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:27.209371 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c4ade247-ed3a-4156-80db-58fa4198cbba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:27.211297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1011/2023] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:27 2022] DELETE /identity/v3/projects/42e0babfb4c14e3a834da6b8fa45b9e9 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:27.215098 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f5214-ca7a-421e-80dd-bc60f60aa204 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:27.226092 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f5214-ca7a-421e-80dd-bc60f60aa204 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:27.295278 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea76f1-ba97-47f9-86f2-61da4c67cfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:27.296310 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea76f1-ba97-47f9-86f2-61da4c67cfc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:27.296601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea76f1-ba97-47f9-86f2-61da4c67cfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:27.296878 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea76f1-ba97-47f9-86f2-61da4c67cfc0 admin admin] PATH_INFO: `/v3/users/438b0bebb2634efca6f2e0c537a97786` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:27.298069 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea76f1-ba97-47f9-86f2-61da4c67cfc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=438b0bebb2634efca6f2e0c537a97786)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:27.300015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea76f1-ba97-47f9-86f2-61da4c67cfc0 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:27.350430 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a2ea76f1-ba97-47f9-86f2-61da4c67cfc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '438b0bebb2634efca6f2e0c537a97786'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:27.359766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a2ea76f1-ba97-47f9-86f2-61da4c67cfc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '438b0bebb2634efca6f2e0c537a97786'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:27.368474 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a2ea76f1-ba97-47f9-86f2-61da4c67cfc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '438b0bebb2634efca6f2e0c537a97786'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:27.369413 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a2ea76f1-ba97-47f9-86f2-61da4c67cfc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '438b0bebb2634efca6f2e0c537a97786'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:27.391827 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a2ea76f1-ba97-47f9-86f2-61da4c67cfc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '438b0bebb2634efca6f2e0c537a97786'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:27.401570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1013/2024] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:27 2022] DELETE /identity/v3/users/438b0bebb2634efca6f2e0c537a97786 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:27.885023 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ade247-ed3a-4156-80db-58fa4198cbba admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:27.886577 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c4ade247-ed3a-4156-80db-58fa4198cbba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:27.907270 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ade247-ed3a-4156-80db-58fa4198cbba admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:28.007927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5caf7-25a2-451c-afb8-6690e609c6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:28.008927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5caf7-25a2-451c-afb8-6690e609c6fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:28.009260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5caf7-25a2-451c-afb8-6690e609c6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:28.009788 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5caf7-25a2-451c-afb8-6690e609c6fd admin admin] PATH_INFO: `/v3/projects/a66d037cd93541a68bd32e178c3448eb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:28.011511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5caf7-25a2-451c-afb8-6690e609c6fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a66d037cd93541a68bd32e178c3448eb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:28.014225 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5caf7-25a2-451c-afb8-6690e609c6fd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:28.045616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:28.080239 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9da5caf7-25a2-451c-afb8-6690e609c6fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a66d037cd93541a68bd32e178c3448eb', 'request_id': 'req-9da5caf7-25a2-451c-afb8-6690e609c6fd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:28.093048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9da5caf7-25a2-451c-afb8-6690e609c6fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a66d037cd93541a68bd32e178c3448eb', 'request_id': 'req-9da5caf7-25a2-451c-afb8-6690e609c6fd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:28.106742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9da5caf7-25a2-451c-afb8-6690e609c6fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a66d037cd93541a68bd32e178c3448eb', 'request_id': 'req-9da5caf7-25a2-451c-afb8-6690e609c6fd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:28.129442 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9da5caf7-25a2-451c-afb8-6690e609c6fd admin admin] The token cache is being invalidate because project a66d037cd93541a68bd32e178c3448eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:28.129799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9da5caf7-25a2-451c-afb8-6690e609c6fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:28.137173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9da5caf7-25a2-451c-afb8-6690e609c6fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:28.142163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1012/2025] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:27 2022] DELETE /identity/v3/projects/a66d037cd93541a68bd32e178c3448eb => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:28.143628 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea76f1-ba97-47f9-86f2-61da4c67cfc0 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:28.145354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a2ea76f1-ba97-47f9-86f2-61da4c67cfc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:28.161764 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea76f1-ba97-47f9-86f2-61da4c67cfc0 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:28.250201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d249406f-761f-43f5-92e3-62cf786f6bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:28.251204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249406f-761f-43f5-92e3-62cf786f6bcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:28.251976 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249406f-761f-43f5-92e3-62cf786f6bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:28.251976 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249406f-761f-43f5-92e3-62cf786f6bcf admin admin] PATH_INFO: `/v3/users/af3fd7c71ee74e0b94d5b2bfc84cb69d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:28.253506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d249406f-761f-43f5-92e3-62cf786f6bcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=af3fd7c71ee74e0b94d5b2bfc84cb69d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:28.255799 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d249406f-761f-43f5-92e3-62cf786f6bcf admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:28.282178 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:28.335440 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d249406f-761f-43f5-92e3-62cf786f6bcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af3fd7c71ee74e0b94d5b2bfc84cb69d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:28.345264 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d249406f-761f-43f5-92e3-62cf786f6bcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af3fd7c71ee74e0b94d5b2bfc84cb69d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:28.354142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d249406f-761f-43f5-92e3-62cf786f6bcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af3fd7c71ee74e0b94d5b2bfc84cb69d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:28.354848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d249406f-761f-43f5-92e3-62cf786f6bcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af3fd7c71ee74e0b94d5b2bfc84cb69d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:28.370062 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d249406f-761f-43f5-92e3-62cf786f6bcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af3fd7c71ee74e0b94d5b2bfc84cb69d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:28.378573 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1014/2026] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:28 2022] DELETE /identity/v3/users/af3fd7c71ee74e0b94d5b2bfc84cb69d => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:28.668133 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5caf7-25a2-451c-afb8-6690e609c6fd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:28.669659 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9da5caf7-25a2-451c-afb8-6690e609c6fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:28.698892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5caf7-25a2-451c-afb8-6690e609c6fd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:28.774527 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f936bfa-e7a8-4188-a147-f5f6a92deb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:28.775408 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f936bfa-e7a8-4188-a147-f5f6a92deb0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:28.775894 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f936bfa-e7a8-4188-a147-f5f6a92deb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:28.776213 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f936bfa-e7a8-4188-a147-f5f6a92deb0a admin admin] PATH_INFO: `/v3/projects/cb688abaca0a4e6d8a57334e3d37a285` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:28.777618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f936bfa-e7a8-4188-a147-f5f6a92deb0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb688abaca0a4e6d8a57334e3d37a285)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:28.779792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f936bfa-e7a8-4188-a147-f5f6a92deb0a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:28.844537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6f936bfa-e7a8-4188-a147-f5f6a92deb0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb688abaca0a4e6d8a57334e3d37a285', 'request_id': 'req-6f936bfa-e7a8-4188-a147-f5f6a92deb0a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:28.853777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6f936bfa-e7a8-4188-a147-f5f6a92deb0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb688abaca0a4e6d8a57334e3d37a285', 'request_id': 'req-6f936bfa-e7a8-4188-a147-f5f6a92deb0a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:28.862807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6f936bfa-e7a8-4188-a147-f5f6a92deb0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb688abaca0a4e6d8a57334e3d37a285', 'request_id': 'req-6f936bfa-e7a8-4188-a147-f5f6a92deb0a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:28.879255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6f936bfa-e7a8-4188-a147-f5f6a92deb0a admin admin] The token cache is being invalidate because project cb688abaca0a4e6d8a57334e3d37a285 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:28.879718 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6f936bfa-e7a8-4188-a147-f5f6a92deb0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:28.888592 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6f936bfa-e7a8-4188-a147-f5f6a92deb0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:28.891060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1013/2027] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:28 2022] DELETE /identity/v3/projects/cb688abaca0a4e6d8a57334e3d37a285 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:28.897030 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2d129-a554-4c7f-aece-189618791807 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:28.897555 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2d129-a554-4c7f-aece-189618791807 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:28.898410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d2d129-a554-4c7f-aece-189618791807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:28.917523 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c2d2d129-a554-4c7f-aece-189618791807 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:28.919973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c2d2d129-a554-4c7f-aece-189618791807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:28.990229 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1015/2028] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:29.001004 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e13eb-0ede-4ba6-84a9-a31e061123b8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:29.001302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e13eb-0ede-4ba6-84a9-a31e061123b8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:29.001592 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e13eb-0ede-4ba6-84a9-a31e061123b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:29.024359 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8c9e13eb-0ede-4ba6-84a9-a31e061123b8 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:29.025804 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8c9e13eb-0ede-4ba6-84a9-a31e061123b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:29.034329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1014/2029] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:29.039214 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2d129-a554-4c7f-aece-189618791807 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:29.049291 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d2d129-a554-4c7f-aece-189618791807 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:29.054191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8518d-b7fb-4626-92a9-12525ecb93a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:29.054973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8518d-b7fb-4626-92a9-12525ecb93a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:29.055366 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8518d-b7fb-4626-92a9-12525ecb93a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:29.055849 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8518d-b7fb-4626-92a9-12525ecb93a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:29.058980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8518d-b7fb-4626-92a9-12525ecb93a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:29.059545 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8518d-b7fb-4626-92a9-12525ecb93a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:29.061373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8518d-b7fb-4626-92a9-12525ecb93a0 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:29.079264 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1016/2030] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:29.080460 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e13eb-0ede-4ba6-84a9-a31e061123b8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:29.092542 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e13eb-0ede-4ba6-84a9-a31e061123b8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:29.097813 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11888cb-ed4b-458e-a36f-a168e82c704c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:29.100503 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11888cb-ed4b-458e-a36f-a168e82c704c admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:29.101044 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11888cb-ed4b-458e-a36f-a168e82c704c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:29.102323 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11888cb-ed4b-458e-a36f-a168e82c704c admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:29.103242 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11888cb-ed4b-458e-a36f-a168e82c704c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:29.105996 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11888cb-ed4b-458e-a36f-a168e82c704c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:29.140137 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1015/2031] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:29 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:29.144042 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8518d-b7fb-4626-92a9-12525ecb93a0 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:29.162171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8518d-b7fb-4626-92a9-12525ecb93a0 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:29.235512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af2060-9967-472e-b56a-157f3708aa94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:29.236442 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af2060-9967-472e-b56a-157f3708aa94 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:29.236819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af2060-9967-472e-b56a-157f3708aa94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:29.237206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af2060-9967-472e-b56a-157f3708aa94 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:29.238021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af2060-9967-472e-b56a-157f3708aa94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:29.240231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af2060-9967-472e-b56a-157f3708aa94 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:29.264274 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1017/2032] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:29 2022] POST /identity/v3/users => generated 294 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:29.269015 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11888cb-ed4b-458e-a36f-a168e82c704c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:29.281565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11888cb-ed4b-458e-a36f-a168e82c704c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:29.286458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c228539-01f7-4e45-8334-746e961dc9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:29.287141 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c228539-01f7-4e45-8334-746e961dc9a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:29.287416 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c228539-01f7-4e45-8334-746e961dc9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:29.287698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c228539-01f7-4e45-8334-746e961dc9a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:29.288963 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c228539-01f7-4e45-8334-746e961dc9a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:29.290604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c228539-01f7-4e45-8334-746e961dc9a3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:29.292996 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:29.304971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1016/2033] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:09:29.308755 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af2060-9967-472e-b56a-157f3708aa94 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:29.320019 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af2060-9967-472e-b56a-157f3708aa94 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:29.324859 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3a824-21d8-44bb-ac37-9857c3481315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:29.325635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3a824-21d8-44bb-ac37-9857c3481315 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:29.325997 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3a824-21d8-44bb-ac37-9857c3481315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:29.326361 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3a824-21d8-44bb-ac37-9857c3481315 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:29.327167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3a824-21d8-44bb-ac37-9857c3481315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:29.328901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3a824-21d8-44bb-ac37-9857c3481315 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:29.334413 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:29.342165 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1018/2034] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:29.345647 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c228539-01f7-4e45-8334-746e961dc9a3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:29.356571 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c228539-01f7-4e45-8334-746e961dc9a3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:29.361196 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea40b4-e39c-4c91-9b89-9fe2b33a2e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:29.362079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea40b4-e39c-4c91-9b89-9fe2b33a2e12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:29.362364 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea40b4-e39c-4c91-9b89-9fe2b33a2e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:29.362642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea40b4-e39c-4c91-9b89-9fe2b33a2e12 admin admin] PATH_INFO: `/v3/projects/2d39e2f24b24421d85f727d708494b11/users/cb5f02eef6f448239b7aea257d8e9b01/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:29.378218 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea40b4-e39c-4c91-9b89-9fe2b33a2e12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d39e2f24b24421d85f727d708494b11, user_id=cb5f02eef6f448239b7aea257d8e9b01, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:29.380932 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eea40b4-e39c-4c91-9b89-9fe2b33a2e12 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:29.396110 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1017/2035] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:29 2022] PUT /identity/v3/projects/2d39e2f24b24421d85f727d708494b11/users/cb5f02eef6f448239b7aea257d8e9b01/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:29.407795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf89bb-cbf4-4a1b-ac46-7e83a5cd132e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:29.408311 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf89bb-cbf4-4a1b-ac46-7e83a5cd132e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:29.408591 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf89bb-cbf4-4a1b-ac46-7e83a5cd132e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:29.436934 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-cadf89bb-cbf4-4a1b-ac46-7e83a5cd132e None None] MFA Rules not processed for user `cb5f02eef6f448239b7aea257d8e9b01`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:29.438368 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-cadf89bb-cbf4-4a1b-ac46-7e83a5cd132e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:29.574150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1019/2036] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:29.574819 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea40b4-e39c-4c91-9b89-9fe2b33a2e12 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:29.624279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf89bb-cbf4-4a1b-ac46-7e83a5cd132e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:29.625962 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-cadf89bb-cbf4-4a1b-ac46-7e83a5cd132e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:29.654341 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea40b4-e39c-4c91-9b89-9fe2b33a2e12 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:29.659272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c42c2-a88b-462b-9a29-72cae321dbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:29.660000 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c42c2-a88b-462b-9a29-72cae321dbc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:29.660296 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c42c2-a88b-462b-9a29-72cae321dbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:29.660579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c42c2-a88b-462b-9a29-72cae321dbc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:29.661294 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c42c2-a88b-462b-9a29-72cae321dbc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:29.663250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c42c2-a88b-462b-9a29-72cae321dbc2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:29.693261 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1018/2037] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:29 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:09:29.697323 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c42c2-a88b-462b-9a29-72cae321dbc2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:29.703348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf89bb-cbf4-4a1b-ac46-7e83a5cd132e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:29.771258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a01298-3f76-43f0-9d67-c95dbdecc60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:29.772183 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a01298-3f76-43f0-9d67-c95dbdecc60c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:29.772972 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a01298-3f76-43f0-9d67-c95dbdecc60c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:29.773271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a01298-3f76-43f0-9d67-c95dbdecc60c admin admin] PATH_INFO: `/v3/users/2c6a5983bee24f438becc13b28bd0405` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:29.774539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a01298-3f76-43f0-9d67-c95dbdecc60c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c6a5983bee24f438becc13b28bd0405)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:29.776602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a01298-3f76-43f0-9d67-c95dbdecc60c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:29.786237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c42c2-a88b-462b-9a29-72cae321dbc2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:29.791764 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307e33a-8447-4754-aa40-6d4704d69933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:29.792575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307e33a-8447-4754-aa40-6d4704d69933 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:29.792941 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307e33a-8447-4754-aa40-6d4704d69933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:29.793623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307e33a-8447-4754-aa40-6d4704d69933 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:29.794062 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307e33a-8447-4754-aa40-6d4704d69933 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:29.796111 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307e33a-8447-4754-aa40-6d4704d69933 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:29.823237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1019/2038] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:29 2022] POST /identity/v3/users => generated 300 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:09:29.827663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307e33a-8447-4754-aa40-6d4704d69933 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:29.832704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:29.841387 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307e33a-8447-4754-aa40-6d4704d69933 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:29.842115 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-52a01298-3f76-43f0-9d67-c95dbdecc60c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c6a5983bee24f438becc13b28bd0405'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:29.846394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd47af-bb19-4e3b-a3e9-83b90bb6e8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:29.847147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd47af-bb19-4e3b-a3e9-83b90bb6e8b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:29.847460 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd47af-bb19-4e3b-a3e9-83b90bb6e8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:29.847898 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd47af-bb19-4e3b-a3e9-83b90bb6e8b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:29.848931 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd47af-bb19-4e3b-a3e9-83b90bb6e8b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:29.850753 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd47af-bb19-4e3b-a3e9-83b90bb6e8b2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:29.851080 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-52a01298-3f76-43f0-9d67-c95dbdecc60c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c6a5983bee24f438becc13b28bd0405'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:29.859606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-52a01298-3f76-43f0-9d67-c95dbdecc60c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c6a5983bee24f438becc13b28bd0405'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:29.860347 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-52a01298-3f76-43f0-9d67-c95dbdecc60c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c6a5983bee24f438becc13b28bd0405'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:29.864151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1020/2039] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:29.868510 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd47af-bb19-4e3b-a3e9-83b90bb6e8b2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:29.870352 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6dcd47af-bb19-4e3b-a3e9-83b90bb6e8b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:29.874226 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-52a01298-3f76-43f0-9d67-c95dbdecc60c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c6a5983bee24f438becc13b28bd0405'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:29.881095 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1020/2040] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:29 2022] DELETE /identity/v3/users/2c6a5983bee24f438becc13b28bd0405 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:29.886410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62f6a0-6fee-49e5-9fe4-8cc79d1e2b54 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:29.886785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62f6a0-6fee-49e5-9fe4-8cc79d1e2b54 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:29.887062 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62f6a0-6fee-49e5-9fe4-8cc79d1e2b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:29.887866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd47af-bb19-4e3b-a3e9-83b90bb6e8b2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:29.909957 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-9a62f6a0-6fee-49e5-9fe4-8cc79d1e2b54 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:29.911339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9a62f6a0-6fee-49e5-9fe4-8cc79d1e2b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:29.928115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:29.967387 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f53805-d344-474a-b9d9-656497cc4a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:29.969578 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f53805-d344-474a-b9d9-656497cc4a44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:29.969973 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f53805-d344-474a-b9d9-656497cc4a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:29.970389 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f53805-d344-474a-b9d9-656497cc4a44 admin admin] PATH_INFO: `/v3/projects/d8b476afb1934c408a401b6f6344ffa5/users/2d2175db9c364716a458110a95ea0f4f/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:30.006373 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f53805-d344-474a-b9d9-656497cc4a44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8b476afb1934c408a401b6f6344ffa5, user_id=2d2175db9c364716a458110a95ea0f4f, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:30.008221 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1021/2041] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:30.009149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f53805-d344-474a-b9d9-656497cc4a44 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:30.013934 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62f6a0-6fee-49e5-9fe4-8cc79d1e2b54 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:30.015267 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9a62f6a0-6fee-49e5-9fe4-8cc79d1e2b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:30.031458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1021/2042] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:29 2022] PUT /identity/v3/projects/d8b476afb1934c408a401b6f6344ffa5/users/2d2175db9c364716a458110a95ea0f4f/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:30.039589 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f53805-d344-474a-b9d9-656497cc4a44 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:30.092135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62f6a0-6fee-49e5-9fe4-8cc79d1e2b54 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:30.125356 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f53805-d344-474a-b9d9-656497cc4a44 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:30.131517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bba69-e18e-4175-be04-003dc5f6017d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:30.133251 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bba69-e18e-4175-be04-003dc5f6017d admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:30.133579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bba69-e18e-4175-be04-003dc5f6017d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:30.133836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549bba69-e18e-4175-be04-003dc5f6017d admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:30.134583 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549bba69-e18e-4175-be04-003dc5f6017d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:30.136813 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549bba69-e18e-4175-be04-003dc5f6017d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:30.149932 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1022/2043] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:30.154312 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bba69-e18e-4175-be04-003dc5f6017d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:30.176342 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549bba69-e18e-4175-be04-003dc5f6017d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:30.181526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c0ebe-ebe1-4435-930c-df6d874d2973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:30.182205 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c0ebe-ebe1-4435-930c-df6d874d2973 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:30.182579 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c0ebe-ebe1-4435-930c-df6d874d2973 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:30.182828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c0ebe-ebe1-4435-930c-df6d874d2973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:30.187788 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63fbb63-578b-471b-8d15-423d195adea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:30.188721 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fbb63-578b-471b-8d15-423d195adea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:30.189016 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fbb63-578b-471b-8d15-423d195adea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:30.189297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fbb63-578b-471b-8d15-423d195adea8 admin admin] PATH_INFO: `/v3/projects/d8b476afb1934c408a401b6f6344ffa5/users/2d2175db9c364716a458110a95ea0f4f/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:30.192767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63fbb63-578b-471b-8d15-423d195adea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8b476afb1934c408a401b6f6344ffa5, user_id=2d2175db9c364716a458110a95ea0f4f, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:30.197724 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63fbb63-578b-471b-8d15-423d195adea8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:30.211656 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c0ebe-ebe1-4435-930c-df6d874d2973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:30.213172 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c0ebe-ebe1-4435-930c-df6d874d2973 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:30.232505 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1023/2044] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:30 2022] PUT /identity/v3/projects/d8b476afb1934c408a401b6f6344ffa5/users/2d2175db9c364716a458110a95ea0f4f/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:30.232696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1022/2045] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:09:30.232778 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304905f-668a-4561-966f-148709fddaf4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:30.232778 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304905f-668a-4561-966f-148709fddaf4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:30.232778 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304905f-668a-4561-966f-148709fddaf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:30.269683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3304905f-668a-4561-966f-148709fddaf4 None None] MFA Rules not processed for user `2d2175db9c364716a458110a95ea0f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:30.271058 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3304905f-668a-4561-966f-148709fddaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:30.360080 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1024/2046] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:30.373561 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c0ebe-ebe1-4435-930c-df6d874d2973 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:30.374370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fe8c0ebe-ebe1-4435-930c-df6d874d2973 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:30.396343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:30.447180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c0ebe-ebe1-4435-930c-df6d874d2973 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:30.509243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c891415-874f-46d2-bfe4-61155154da62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:30.510021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c891415-874f-46d2-bfe4-61155154da62 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:30.510499 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c891415-874f-46d2-bfe4-61155154da62 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:30.513116 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c891415-874f-46d2-bfe4-61155154da62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:30.526870 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c891415-874f-46d2-bfe4-61155154da62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:30.528610 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c891415-874f-46d2-bfe4-61155154da62 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:30.549004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1023/2047] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:30.694824 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3304905f-668a-4561-966f-148709fddaf4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:30.695590 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c891415-874f-46d2-bfe4-61155154da62 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:30.696927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3304905f-668a-4561-966f-148709fddaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:30.746071 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:30.782309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c891415-874f-46d2-bfe4-61155154da62 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:30.787511 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b5916-06c6-4ccf-a296-42ebc2161df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:30.789113 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b5916-06c6-4ccf-a296-42ebc2161df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:30.789483 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b5916-06c6-4ccf-a296-42ebc2161df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:30.789830 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b5916-06c6-4ccf-a296-42ebc2161df4 admin admin] PATH_INFO: `/v3/users/cb5f02eef6f448239b7aea257d8e9b01` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:30.791159 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b5916-06c6-4ccf-a296-42ebc2161df4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb5f02eef6f448239b7aea257d8e9b01)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:30.793159 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b5916-06c6-4ccf-a296-42ebc2161df4 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:30.793491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3304905f-668a-4561-966f-148709fddaf4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:30.800301 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda76aa-f365-4f62-89d3-4918d065e384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:30.801189 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda76aa-f365-4f62-89d3-4918d065e384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:30.801487 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda76aa-f365-4f62-89d3-4918d065e384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:30.801769 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda76aa-f365-4f62-89d3-4918d065e384 admin admin] PATH_INFO: `/v3/projects/ba65aa2594a24b669af81620d2af7ef5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:30.803027 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda76aa-f365-4f62-89d3-4918d065e384 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba65aa2594a24b669af81620d2af7ef5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:30.806601 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda76aa-f365-4f62-89d3-4918d065e384 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:30.849804 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c89b5916-06c6-4ccf-a296-42ebc2161df4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb5f02eef6f448239b7aea257d8e9b01'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:30.860321 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c89b5916-06c6-4ccf-a296-42ebc2161df4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb5f02eef6f448239b7aea257d8e9b01'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:30.873674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c89b5916-06c6-4ccf-a296-42ebc2161df4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb5f02eef6f448239b7aea257d8e9b01'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:30.874831 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c89b5916-06c6-4ccf-a296-42ebc2161df4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb5f02eef6f448239b7aea257d8e9b01'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:30.878965 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ccda76aa-f365-4f62-89d3-4918d065e384 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba65aa2594a24b669af81620d2af7ef5', 'request_id': 'req-ccda76aa-f365-4f62-89d3-4918d065e384'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:30.888452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ccda76aa-f365-4f62-89d3-4918d065e384 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba65aa2594a24b669af81620d2af7ef5', 'request_id': 'req-ccda76aa-f365-4f62-89d3-4918d065e384'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:30.901094 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:30.905233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ccda76aa-f365-4f62-89d3-4918d065e384 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba65aa2594a24b669af81620d2af7ef5', 'request_id': 'req-ccda76aa-f365-4f62-89d3-4918d065e384'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:30.909504 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c89b5916-06c6-4ccf-a296-42ebc2161df4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb5f02eef6f448239b7aea257d8e9b01'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:30.924097 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ccda76aa-f365-4f62-89d3-4918d065e384 admin admin] The token cache is being invalidate because project ba65aa2594a24b669af81620d2af7ef5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:30.924838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ccda76aa-f365-4f62-89d3-4918d065e384 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:30.925575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ccda76aa-f365-4f62-89d3-4918d065e384 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:30.925872 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1024/2048] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:30 2022] DELETE /identity/v3/users/cb5f02eef6f448239b7aea257d8e9b01 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:30.927787 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1025/2049] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:30 2022] DELETE /identity/v3/projects/ba65aa2594a24b669af81620d2af7ef5 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:30.931441 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32b6a7-baf8-4eaa-b0fd-b85f7664f75b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:30.931747 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32b6a7-baf8-4eaa-b0fd-b85f7664f75b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:30.932052 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda76aa-f365-4f62-89d3-4918d065e384 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:30.932473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32b6a7-baf8-4eaa-b0fd-b85f7664f75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:30.933528 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ccda76aa-f365-4f62-89d3-4918d065e384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:30.958043 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-1c32b6a7-baf8-4eaa-b0fd-b85f7664f75b None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:30.959487 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1c32b6a7-baf8-4eaa-b0fd-b85f7664f75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:31.025949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda76aa-f365-4f62-89d3-4918d065e384 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:31.041958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1025/2050] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:09:31.053215 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32b6a7-baf8-4eaa-b0fd-b85f7664f75b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:31.054674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1c32b6a7-baf8-4eaa-b0fd-b85f7664f75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:31.099105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aea067-16f1-44ce-81a6-b86dc5c1d75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:31.100051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aea067-16f1-44ce-81a6-b86dc5c1d75f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:31.100426 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aea067-16f1-44ce-81a6-b86dc5c1d75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:31.101347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aea067-16f1-44ce-81a6-b86dc5c1d75f admin admin] PATH_INFO: `/v3/users/49bfbbc1612e467b808e7a9acddab665` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:31.104886 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aea067-16f1-44ce-81a6-b86dc5c1d75f admin admin] RBAC: Authorizing `identity:delete_user(user_id=49bfbbc1612e467b808e7a9acddab665)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:31.107034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aea067-16f1-44ce-81a6-b86dc5c1d75f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:31.113472 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:31.122180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32b6a7-baf8-4eaa-b0fd-b85f7664f75b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:31.126692 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdf9c4-8354-44e1-ba5e-f366bf239036 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:31.127420 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdf9c4-8354-44e1-ba5e-f366bf239036 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:31.127780 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdf9c4-8354-44e1-ba5e-f366bf239036 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:31.128171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdf9c4-8354-44e1-ba5e-f366bf239036 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:31.139467 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdf9c4-8354-44e1-ba5e-f366bf239036 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:31.141678 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdf9c4-8354-44e1-ba5e-f366bf239036 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:31.153432 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1026/2051] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:31.183377 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-98aea067-16f1-44ce-81a6-b86dc5c1d75f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49bfbbc1612e467b808e7a9acddab665'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:31.198741 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-98aea067-16f1-44ce-81a6-b86dc5c1d75f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49bfbbc1612e467b808e7a9acddab665'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:31.208878 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-98aea067-16f1-44ce-81a6-b86dc5c1d75f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49bfbbc1612e467b808e7a9acddab665'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:31.210738 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-98aea067-16f1-44ce-81a6-b86dc5c1d75f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49bfbbc1612e467b808e7a9acddab665'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:31.232958 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-98aea067-16f1-44ce-81a6-b86dc5c1d75f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49bfbbc1612e467b808e7a9acddab665'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:31.242047 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1026/2052] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:30 2022] DELETE /identity/v3/users/49bfbbc1612e467b808e7a9acddab665 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:31.529474 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdf9c4-8354-44e1-ba5e-f366bf239036 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:31.531315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3bcdf9c4-8354-44e1-ba5e-f366bf239036 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:31.573798 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdf9c4-8354-44e1-ba5e-f366bf239036 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:31.705818 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c00a56-0171-4af5-82d9-8256dfb4b5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:31.706788 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c00a56-0171-4af5-82d9-8256dfb4b5b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:31.707154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c00a56-0171-4af5-82d9-8256dfb4b5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:31.707869 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c00a56-0171-4af5-82d9-8256dfb4b5b0 admin admin] PATH_INFO: `/v3/projects/2d39e2f24b24421d85f727d708494b11` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:31.709660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c00a56-0171-4af5-82d9-8256dfb4b5b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d39e2f24b24421d85f727d708494b11)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:31.712608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c00a56-0171-4af5-82d9-8256dfb4b5b0 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:31.789236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:DELETE=4,keystone:SELECT=96 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:31.790492 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aea067-16f1-44ce-81a6-b86dc5c1d75f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:31.792590 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-98aea067-16f1-44ce-81a6-b86dc5c1d75f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:31.817337 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c3c00a56-0171-4af5-82d9-8256dfb4b5b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d39e2f24b24421d85f727d708494b11', 'request_id': 'req-c3c00a56-0171-4af5-82d9-8256dfb4b5b0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:31.829805 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c3c00a56-0171-4af5-82d9-8256dfb4b5b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d39e2f24b24421d85f727d708494b11', 'request_id': 'req-c3c00a56-0171-4af5-82d9-8256dfb4b5b0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:31.834823 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aea067-16f1-44ce-81a6-b86dc5c1d75f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:31.852741 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c3c00a56-0171-4af5-82d9-8256dfb4b5b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d39e2f24b24421d85f727d708494b11', 'request_id': 'req-c3c00a56-0171-4af5-82d9-8256dfb4b5b0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:31.870067 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c3c00a56-0171-4af5-82d9-8256dfb4b5b0 admin admin] The token cache is being invalidate because project 2d39e2f24b24421d85f727d708494b11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:31.870381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c3c00a56-0171-4af5-82d9-8256dfb4b5b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:31.871016 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c3c00a56-0171-4af5-82d9-8256dfb4b5b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:31.876094 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1027/2053] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:31 2022] DELETE /identity/v3/projects/2d39e2f24b24421d85f727d708494b11 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:31.876652 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c00a56-0171-4af5-82d9-8256dfb4b5b0 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:31.878243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c3c00a56-0171-4af5-82d9-8256dfb4b5b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:31.924686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531476ee-8d9e-441a-8776-964996abceff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:31.925752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531476ee-8d9e-441a-8776-964996abceff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:31.926043 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531476ee-8d9e-441a-8776-964996abceff admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:31.926327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531476ee-8d9e-441a-8776-964996abceff admin admin] PATH_INFO: `/v3/projects/d9e5ffa48c7d4b1188d1daf826efcfaf` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:31.927626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531476ee-8d9e-441a-8776-964996abceff admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9e5ffa48c7d4b1188d1daf826efcfaf)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:31.942438 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531476ee-8d9e-441a-8776-964996abceff admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:31.957881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c00a56-0171-4af5-82d9-8256dfb4b5b0 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:31.964168 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46814b94-d1ee-4244-825d-4ecea583d428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:31.965048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46814b94-d1ee-4244-825d-4ecea583d428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:31.965338 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46814b94-d1ee-4244-825d-4ecea583d428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:31.965615 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46814b94-d1ee-4244-825d-4ecea583d428 admin admin] PATH_INFO: `/v3/users/2d2175db9c364716a458110a95ea0f4f` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:31.966802 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46814b94-d1ee-4244-825d-4ecea583d428 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d2175db9c364716a458110a95ea0f4f)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:31.968789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46814b94-d1ee-4244-825d-4ecea583d428 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:31.998387 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-531476ee-8d9e-441a-8776-964996abceff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9e5ffa48c7d4b1188d1daf826efcfaf', 'request_id': 'req-531476ee-8d9e-441a-8776-964996abceff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:32.005648 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-531476ee-8d9e-441a-8776-964996abceff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9e5ffa48c7d4b1188d1daf826efcfaf', 'request_id': 'req-531476ee-8d9e-441a-8776-964996abceff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:32.015152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-531476ee-8d9e-441a-8776-964996abceff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9e5ffa48c7d4b1188d1daf826efcfaf', 'request_id': 'req-531476ee-8d9e-441a-8776-964996abceff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:32.021121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-46814b94-d1ee-4244-825d-4ecea583d428 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d2175db9c364716a458110a95ea0f4f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:32.026607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-531476ee-8d9e-441a-8776-964996abceff admin admin] The token cache is being invalidate because project d9e5ffa48c7d4b1188d1daf826efcfaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:32.026982 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-531476ee-8d9e-441a-8776-964996abceff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:32.027772 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-531476ee-8d9e-441a-8776-964996abceff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:32.028887 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-46814b94-d1ee-4244-825d-4ecea583d428 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d2175db9c364716a458110a95ea0f4f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:32.030368 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1027/2054] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:31 2022] DELETE /identity/v3/projects/d9e5ffa48c7d4b1188d1daf826efcfaf => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:32.034170 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531476ee-8d9e-441a-8776-964996abceff admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:32.035584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-531476ee-8d9e-441a-8776-964996abceff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:32.036483 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-46814b94-d1ee-4244-825d-4ecea583d428 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d2175db9c364716a458110a95ea0f4f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:32.037206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-46814b94-d1ee-4244-825d-4ecea583d428 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d2175db9c364716a458110a95ea0f4f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:32.050278 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-46814b94-d1ee-4244-825d-4ecea583d428 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d2175db9c364716a458110a95ea0f4f'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:32.067511 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1028/2055] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:31 2022] DELETE /identity/v3/users/2d2175db9c364716a458110a95ea0f4f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:32.068299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:32.118618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531476ee-8d9e-441a-8776-964996abceff admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:32.180157 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfb10b-7905-4076-8583-55df04d43bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:32.181113 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfb10b-7905-4076-8583-55df04d43bb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:32.181408 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfb10b-7905-4076-8583-55df04d43bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:32.181751 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfb10b-7905-4076-8583-55df04d43bb3 admin admin] PATH_INFO: `/v3/users/3f1f3423f24f45f48398900c1639af36` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:32.182942 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bfb10b-7905-4076-8583-55df04d43bb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f1f3423f24f45f48398900c1639af36)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:32.184911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bfb10b-7905-4076-8583-55df04d43bb3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:32.235887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-57bfb10b-7905-4076-8583-55df04d43bb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f1f3423f24f45f48398900c1639af36'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:32.246166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-57bfb10b-7905-4076-8583-55df04d43bb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f1f3423f24f45f48398900c1639af36'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:32.257422 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-57bfb10b-7905-4076-8583-55df04d43bb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f1f3423f24f45f48398900c1639af36'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:32.258086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-57bfb10b-7905-4076-8583-55df04d43bb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f1f3423f24f45f48398900c1639af36'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:32.272195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-57bfb10b-7905-4076-8583-55df04d43bb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f1f3423f24f45f48398900c1639af36'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:32.278605 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1028/2056] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:32 2022] DELETE /identity/v3/users/3f1f3423f24f45f48398900c1639af36 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:32.370344 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46814b94-d1ee-4244-825d-4ecea583d428 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:32.372275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-46814b94-d1ee-4244-825d-4ecea583d428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:32.403735 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46814b94-d1ee-4244-825d-4ecea583d428 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:32.462481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:32.479312 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c370b68-1c0e-4e42-868f-5fe8118cf895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:32.480225 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c370b68-1c0e-4e42-868f-5fe8118cf895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:32.480520 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c370b68-1c0e-4e42-868f-5fe8118cf895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:32.480807 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c370b68-1c0e-4e42-868f-5fe8118cf895 admin admin] PATH_INFO: `/v3/projects/d8b476afb1934c408a401b6f6344ffa5` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:32.482098 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c370b68-1c0e-4e42-868f-5fe8118cf895 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8b476afb1934c408a401b6f6344ffa5)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:32.484069 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c370b68-1c0e-4e42-868f-5fe8118cf895 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:32.543203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6c370b68-1c0e-4e42-868f-5fe8118cf895 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8b476afb1934c408a401b6f6344ffa5', 'request_id': 'req-6c370b68-1c0e-4e42-868f-5fe8118cf895'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:32.548743 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6c370b68-1c0e-4e42-868f-5fe8118cf895 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8b476afb1934c408a401b6f6344ffa5', 'request_id': 'req-6c370b68-1c0e-4e42-868f-5fe8118cf895'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:32.557977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6c370b68-1c0e-4e42-868f-5fe8118cf895 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8b476afb1934c408a401b6f6344ffa5', 'request_id': 'req-6c370b68-1c0e-4e42-868f-5fe8118cf895'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:32.570334 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6c370b68-1c0e-4e42-868f-5fe8118cf895 admin admin] The token cache is being invalidate because project d8b476afb1934c408a401b6f6344ffa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:32.570633 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6c370b68-1c0e-4e42-868f-5fe8118cf895 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:32.571311 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6c370b68-1c0e-4e42-868f-5fe8118cf895 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:32.572901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1029/2057] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:32 2022] DELETE /identity/v3/projects/d8b476afb1934c408a401b6f6344ffa5 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:32.578400 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922cb52-d475-4ff9-a452-66d22fcb0409 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:32.578400 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922cb52-d475-4ff9-a452-66d22fcb0409 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:32.578688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922cb52-d475-4ff9-a452-66d22fcb0409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:32.600134 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d922cb52-d475-4ff9-a452-66d22fcb0409 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:32.602079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d922cb52-d475-4ff9-a452-66d22fcb0409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:32.641410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:32.667516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1029/2058] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:32.677796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851e477-9814-4c66-b72f-7bbbc4168c43 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:32.678086 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851e477-9814-4c66-b72f-7bbbc4168c43 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:32.678367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851e477-9814-4c66-b72f-7bbbc4168c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:32.697843 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-e851e477-9814-4c66-b72f-7bbbc4168c43 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:32.699076 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e851e477-9814-4c66-b72f-7bbbc4168c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:32.709481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1030/2059] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:32.714487 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922cb52-d475-4ff9-a452-66d22fcb0409 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:32.724667 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922cb52-d475-4ff9-a452-66d22fcb0409 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:32.729961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc754e-519b-4c8a-b636-117501466159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:32.730642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc754e-519b-4c8a-b636-117501466159 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:32.730932 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc754e-519b-4c8a-b636-117501466159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:32.731213 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc754e-519b-4c8a-b636-117501466159 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:32.731987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc754e-519b-4c8a-b636-117501466159 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:32.732304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc754e-519b-4c8a-b636-117501466159 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:32.733932 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc754e-519b-4c8a-b636-117501466159 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:32.752147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1030/2060] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:32.754497 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851e477-9814-4c66-b72f-7bbbc4168c43 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:32.765187 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851e477-9814-4c66-b72f-7bbbc4168c43 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:32.769885 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df568a-4303-4bfc-becc-0ee9ffd28646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:32.770555 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df568a-4303-4bfc-becc-0ee9ffd28646 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:32.770839 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df568a-4303-4bfc-becc-0ee9ffd28646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:32.771116 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df568a-4303-4bfc-becc-0ee9ffd28646 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:32.771815 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df568a-4303-4bfc-becc-0ee9ffd28646 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:32.773782 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df568a-4303-4bfc-becc-0ee9ffd28646 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:32.805404 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1031/2061] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:32 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:32.806741 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc754e-519b-4c8a-b636-117501466159 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:32.824907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc754e-519b-4c8a-b636-117501466159 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:32.886098 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98832d2-908b-491b-9019-62a2ac81f29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:32.886968 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98832d2-908b-491b-9019-62a2ac81f29a admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:32.887476 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98832d2-908b-491b-9019-62a2ac81f29a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:32.888151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98832d2-908b-491b-9019-62a2ac81f29a admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:32.888947 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98832d2-908b-491b-9019-62a2ac81f29a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:32.890905 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98832d2-908b-491b-9019-62a2ac81f29a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:32.916439 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1031/2062] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:32 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:32.917640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df568a-4303-4bfc-becc-0ee9ffd28646 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:32.928201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df568a-4303-4bfc-becc-0ee9ffd28646 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:32.933433 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca3d80-4613-40a1-b93d-3c822dd7990b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:32.934095 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca3d80-4613-40a1-b93d-3c822dd7990b admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:32.934382 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca3d80-4613-40a1-b93d-3c822dd7990b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:32.934662 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca3d80-4613-40a1-b93d-3c822dd7990b admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:32.935399 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca3d80-4613-40a1-b93d-3c822dd7990b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:32.937125 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca3d80-4613-40a1-b93d-3c822dd7990b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:32.954085 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1032/2063] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:32.958378 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98832d2-908b-491b-9019-62a2ac81f29a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:32.971483 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98832d2-908b-491b-9019-62a2ac81f29a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:32.986213 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8def5d0-a64c-49ec-bbf3-fed2cf77bf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:32.987524 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8def5d0-a64c-49ec-bbf3-fed2cf77bf54 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:32.987524 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8def5d0-a64c-49ec-bbf3-fed2cf77bf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:32.987735 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8def5d0-a64c-49ec-bbf3-fed2cf77bf54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:32.988558 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8def5d0-a64c-49ec-bbf3-fed2cf77bf54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:32.992443 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8def5d0-a64c-49ec-bbf3-fed2cf77bf54 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:33.008932 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1032/2064] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:09:33.009913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca3d80-4613-40a1-b93d-3c822dd7990b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:33.024243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca3d80-4613-40a1-b93d-3c822dd7990b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:33.033326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef5432-3825-47fe-9ec7-f0b863b7faa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:33.034241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef5432-3825-47fe-9ec7-f0b863b7faa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:33.034553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef5432-3825-47fe-9ec7-f0b863b7faa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:33.034860 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef5432-3825-47fe-9ec7-f0b863b7faa0 admin admin] PATH_INFO: `/v3/projects/709e240cf7004f93ad50da59df086482/users/b0cbdf0853db4442be13457bf905b19c/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:33.061505 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef5432-3825-47fe-9ec7-f0b863b7faa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=709e240cf7004f93ad50da59df086482, user_id=b0cbdf0853db4442be13457bf905b19c, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:33.075109 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef5432-3825-47fe-9ec7-f0b863b7faa0 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:33.097140 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1033/2065] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:33 2022] PUT /identity/v3/projects/709e240cf7004f93ad50da59df086482/users/b0cbdf0853db4442be13457bf905b19c/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:33.109450 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a45a93-0edb-4c2d-8d7f-d1eae4383ea1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:33.109450 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a45a93-0edb-4c2d-8d7f-d1eae4383ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:33.109450 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a45a93-0edb-4c2d-8d7f-d1eae4383ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:33.146831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-31a45a93-0edb-4c2d-8d7f-d1eae4383ea1 None None] MFA Rules not processed for user `b0cbdf0853db4442be13457bf905b19c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:33.149130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-31a45a93-0edb-4c2d-8d7f-d1eae4383ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:33.195799 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef5432-3825-47fe-9ec7-f0b863b7faa0 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:33.198120 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-36ef5432-3825-47fe-9ec7-f0b863b7faa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:33.287659 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef5432-3825-47fe-9ec7-f0b863b7faa0 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:33.294469 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a82869-da3d-474f-8e46-0473f4ff5f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:33.295615 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a82869-da3d-474f-8e46-0473f4ff5f18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:33.295906 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a82869-da3d-474f-8e46-0473f4ff5f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:33.296190 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1033/2066] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:33.296698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a82869-da3d-474f-8e46-0473f4ff5f18 admin admin] PATH_INFO: `/v3/projects/9445c0f76e3c42d5b146da4c5d63287a` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:33.300640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a45a93-0edb-4c2d-8d7f-d1eae4383ea1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:33.300881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a82869-da3d-474f-8e46-0473f4ff5f18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9445c0f76e3c42d5b146da4c5d63287a)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:33.313984 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a82869-da3d-474f-8e46-0473f4ff5f18 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:33.328638 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a45a93-0edb-4c2d-8d7f-d1eae4383ea1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:33.337948 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0731f-1f21-47f3-b1ae-457455275619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:33.338630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0731f-1f21-47f3-b1ae-457455275619 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:33.338946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0731f-1f21-47f3-b1ae-457455275619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:33.339230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0731f-1f21-47f3-b1ae-457455275619 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:33.340027 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0731f-1f21-47f3-b1ae-457455275619 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:33.342564 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0731f-1f21-47f3-b1ae-457455275619 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:33.351542 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:33.383993 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1034/2067] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:33 2022] POST /identity/v3/projects => generated 381 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:33.391435 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0731f-1f21-47f3-b1ae-457455275619 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:33.416349 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:33.419905 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-49a82869-da3d-474f-8e46-0473f4ff5f18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9445c0f76e3c42d5b146da4c5d63287a', 'request_id': 'req-49a82869-da3d-474f-8e46-0473f4ff5f18'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:33.428091 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-49a82869-da3d-474f-8e46-0473f4ff5f18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9445c0f76e3c42d5b146da4c5d63287a', 'request_id': 'req-49a82869-da3d-474f-8e46-0473f4ff5f18'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:33.439402 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-49a82869-da3d-474f-8e46-0473f4ff5f18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9445c0f76e3c42d5b146da4c5d63287a', 'request_id': 'req-49a82869-da3d-474f-8e46-0473f4ff5f18'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:33.459098 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-49a82869-da3d-474f-8e46-0473f4ff5f18 admin admin] The token cache is being invalidate because project 9445c0f76e3c42d5b146da4c5d63287a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:33.459408 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-49a82869-da3d-474f-8e46-0473f4ff5f18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:33.461056 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-49a82869-da3d-474f-8e46-0473f4ff5f18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:33.463493 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1034/2068] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:33 2022] DELETE /identity/v3/projects/9445c0f76e3c42d5b146da4c5d63287a => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:33.469311 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f5be9-e735-47c9-9a7c-6126afa2ddc2 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:33.469796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f5be9-e735-47c9-9a7c-6126afa2ddc2 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:33.470187 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f5be9-e735-47c9-9a7c-6126afa2ddc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:33.497438 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0731f-1f21-47f3-b1ae-457455275619 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:33.506444 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f80f5be9-e735-47c9-9a7c-6126afa2ddc2 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:33.508290 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f80f5be9-e735-47c9-9a7c-6126afa2ddc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:33.565800 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b97888-0fea-4ec9-808b-79ab22a0624f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:33.566421 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b97888-0fea-4ec9-808b-79ab22a0624f admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:33.566732 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b97888-0fea-4ec9-808b-79ab22a0624f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:33.567014 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b97888-0fea-4ec9-808b-79ab22a0624f admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:33.567755 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b97888-0fea-4ec9-808b-79ab22a0624f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:33.570405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b97888-0fea-4ec9-808b-79ab22a0624f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:33.584669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1035/2069] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:09:33.592790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1035/2070] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:33 2022] POST /identity/v3/users => generated 297 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:33.594265 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7864e92-cc5f-44ae-b4c4-7c0591727a78 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:33.594569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7864e92-cc5f-44ae-b4c4-7c0591727a78 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:33.594860 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7864e92-cc5f-44ae-b4c4-7c0591727a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:33.596458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b97888-0fea-4ec9-808b-79ab22a0624f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:33.598796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-85b97888-0fea-4ec9-808b-79ab22a0624f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:33.612145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b97888-0fea-4ec9-808b-79ab22a0624f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:33.616903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4891eec4-606c-4232-b32d-93b92dc8ddbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:33.617578 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891eec4-606c-4232-b32d-93b92dc8ddbc admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:33.617867 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891eec4-606c-4232-b32d-93b92dc8ddbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:33.618131 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b7864e92-cc5f-44ae-b4c4-7c0591727a78 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:33.618493 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891eec4-606c-4232-b32d-93b92dc8ddbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:33.619231 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4891eec4-606c-4232-b32d-93b92dc8ddbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:33.619329 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b7864e92-cc5f-44ae-b4c4-7c0591727a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:33.620952 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4891eec4-606c-4232-b32d-93b92dc8ddbc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:33.629788 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1036/2071] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:33.634941 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1036/2072] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:09:33.636028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7864e92-cc5f-44ae-b4c4-7c0591727a78 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:33.636273 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4891eec4-606c-4232-b32d-93b92dc8ddbc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:33.646327 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7864e92-cc5f-44ae-b4c4-7c0591727a78 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:33.649924 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4891eec4-606c-4232-b32d-93b92dc8ddbc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:33.651305 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9602af-a4fe-409d-a441-4d23bd84ea07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:33.652300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9602af-a4fe-409d-a441-4d23bd84ea07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:33.652658 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9602af-a4fe-409d-a441-4d23bd84ea07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:33.653038 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9602af-a4fe-409d-a441-4d23bd84ea07 admin admin] PATH_INFO: `/v3/projects/81b0cffe45524f19b60da7cf01977e77/users/776dc8da9ed84af89b889be47c237955/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:33.655043 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb266b-5366-4c52-b6ca-069907ea0e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:33.655761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb266b-5366-4c52-b6ca-069907ea0e4e admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:33.656142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb266b-5366-4c52-b6ca-069907ea0e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:33.656496 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb266b-5366-4c52-b6ca-069907ea0e4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:33.657297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb266b-5366-4c52-b6ca-069907ea0e4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:33.657676 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb266b-5366-4c52-b6ca-069907ea0e4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:33.659579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb266b-5366-4c52-b6ca-069907ea0e4e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:33.669442 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9602af-a4fe-409d-a441-4d23bd84ea07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81b0cffe45524f19b60da7cf01977e77, user_id=776dc8da9ed84af89b889be47c237955, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:33.672178 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9602af-a4fe-409d-a441-4d23bd84ea07 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:33.677883 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1037/2073] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:33.682640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb266b-5366-4c52-b6ca-069907ea0e4e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:33.701150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1037/2074] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:33 2022] PUT /identity/v3/projects/81b0cffe45524f19b60da7cf01977e77/users/776dc8da9ed84af89b889be47c237955/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:33.701357 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb266b-5366-4c52-b6ca-069907ea0e4e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:33.702688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9602af-a4fe-409d-a441-4d23bd84ea07 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:33.771828 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ed519-2ed1-4ae0-9933-e3988d8bf3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:33.772892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ed519-2ed1-4ae0-9933-e3988d8bf3b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:33.773260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ed519-2ed1-4ae0-9933-e3988d8bf3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:33.773565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ed519-2ed1-4ae0-9933-e3988d8bf3b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:33.774455 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ed519-2ed1-4ae0-9933-e3988d8bf3b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:33.776467 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ed519-2ed1-4ae0-9933-e3988d8bf3b6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:33.778952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9602af-a4fe-409d-a441-4d23bd84ea07 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:33.785000 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf6398-0337-4847-b8e5-b448b62d6794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:33.785680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf6398-0337-4847-b8e5-b448b62d6794 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:33.785970 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf6398-0337-4847-b8e5-b448b62d6794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:33.786258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcf6398-0337-4847-b8e5-b448b62d6794 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:33.787001 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf6398-0337-4847-b8e5-b448b62d6794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:33.789657 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcf6398-0337-4847-b8e5-b448b62d6794 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:33.800897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1038/2075] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:33.805249 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf6398-0337-4847-b8e5-b448b62d6794 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:33.808143 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1038/2076] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:33 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:09:33.812678 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ed519-2ed1-4ae0-9933-e3988d8bf3b6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:33.815859 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcf6398-0337-4847-b8e5-b448b62d6794 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:33.820906 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:33.829509 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:33.885536 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ed519-2ed1-4ae0-9933-e3988d8bf3b6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:33.886202 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca83789-5ce9-4d22-9bc8-e3a6b859b226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:33.887184 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca83789-5ce9-4d22-9bc8-e3a6b859b226 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:33.888539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca83789-5ce9-4d22-9bc8-e3a6b859b226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:33.888908 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca83789-5ce9-4d22-9bc8-e3a6b859b226 admin admin] PATH_INFO: `/v3/projects/81b0cffe45524f19b60da7cf01977e77/users/776dc8da9ed84af89b889be47c237955/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:33.891338 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bcbcf7-bf8e-4aa4-86e1-de43c4034990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:33.892185 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca83789-5ce9-4d22-9bc8-e3a6b859b226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81b0cffe45524f19b60da7cf01977e77, user_id=776dc8da9ed84af89b889be47c237955, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:33.892333 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcbcf7-bf8e-4aa4-86e1-de43c4034990 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:33.892333 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcbcf7-bf8e-4aa4-86e1-de43c4034990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:33.892597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcbcf7-bf8e-4aa4-86e1-de43c4034990 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:33.893293 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bcbcf7-bf8e-4aa4-86e1-de43c4034990 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:33.894931 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca83789-5ce9-4d22-9bc8-e3a6b859b226 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:33.895229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bcbcf7-bf8e-4aa4-86e1-de43c4034990 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:33.906428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1039/2077] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:33 2022] PUT /identity/v3/projects/81b0cffe45524f19b60da7cf01977e77/users/776dc8da9ed84af89b889be47c237955/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:33.917377 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7738a-ba3b-4e42-b1cc-599e762ece7e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:33.917852 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1039/2078] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:33 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:09:33.917974 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7738a-ba3b-4e42-b1cc-599e762ece7e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:33.918431 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7738a-ba3b-4e42-b1cc-599e762ece7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:33.921923 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bcbcf7-bf8e-4aa4-86e1-de43c4034990 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:33.934776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bcbcf7-bf8e-4aa4-86e1-de43c4034990 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:33.973904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f9a7738a-ba3b-4e42-b1cc-599e762ece7e None None] MFA Rules not processed for user `776dc8da9ed84af89b889be47c237955`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:33.976638 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f9a7738a-ba3b-4e42-b1cc-599e762ece7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:34.006893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306325f-2b3d-4c64-a72b-596c60a3e3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:34.007711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306325f-2b3d-4c64-a72b-596c60a3e3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:34.008130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306325f-2b3d-4c64-a72b-596c60a3e3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:34.008482 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306325f-2b3d-4c64-a72b-596c60a3e3f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:34.009287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306325f-2b3d-4c64-a72b-596c60a3e3f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:34.014375 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306325f-2b3d-4c64-a72b-596c60a3e3f8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:34.024589 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1040/2079] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:09:34.028377 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306325f-2b3d-4c64-a72b-596c60a3e3f8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:34.040973 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306325f-2b3d-4c64-a72b-596c60a3e3f8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:34.047285 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29eac4-8557-413c-bba4-db1bed36b079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:34.048632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29eac4-8557-413c-bba4-db1bed36b079 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:34.048632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29eac4-8557-413c-bba4-db1bed36b079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:34.048887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29eac4-8557-413c-bba4-db1bed36b079 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:34.049973 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29eac4-8557-413c-bba4-db1bed36b079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:34.052193 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29eac4-8557-413c-bba4-db1bed36b079 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:34.065728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1041/2080] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:09:34.070601 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29eac4-8557-413c-bba4-db1bed36b079 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:34.079339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1040/2081] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:09:34.080430 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29eac4-8557-413c-bba4-db1bed36b079 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:34.091284 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111e068-67dc-43bf-aef0-17698175b525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:34.091602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7738a-ba3b-4e42-b1cc-599e762ece7e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:34.092243 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111e068-67dc-43bf-aef0-17698175b525 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:34.092530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111e068-67dc-43bf-aef0-17698175b525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:34.096756 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111e068-67dc-43bf-aef0-17698175b525 admin admin] PATH_INFO: `/v3/projects/7312a1e2b25b4a718747bc99c98d64fe/users/a30ecfe7bbe9476cae379d9c08db318d/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:34.096913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f9a7738a-ba3b-4e42-b1cc-599e762ece7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:34.112460 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111e068-67dc-43bf-aef0-17698175b525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7312a1e2b25b4a718747bc99c98d64fe, user_id=a30ecfe7bbe9476cae379d9c08db318d, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:34.115233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111e068-67dc-43bf-aef0-17698175b525 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:34.131507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1042/2082] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:34 2022] PUT /identity/v3/projects/7312a1e2b25b4a718747bc99c98d64fe/users/a30ecfe7bbe9476cae379d9c08db318d/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:34.141868 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a183f-e300-44e4-b6cc-9794024c11ab None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:34.142273 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a183f-e300-44e4-b6cc-9794024c11ab None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:34.142619 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a183f-e300-44e4-b6cc-9794024c11ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:34.156648 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7738a-ba3b-4e42-b1cc-599e762ece7e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:34.171712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cb6a183f-e300-44e4-b6cc-9794024c11ab None None] MFA Rules not processed for user `a30ecfe7bbe9476cae379d9c08db318d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:34.173028 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cb6a183f-e300-44e4-b6cc-9794024c11ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:34.209048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:34.222536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42592e1-fe27-4d42-b92e-42b2650abccd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:34.223212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42592e1-fe27-4d42-b92e-42b2650abccd service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:34.223504 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42592e1-fe27-4d42-b92e-42b2650abccd service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:34.223786 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42592e1-fe27-4d42-b92e-42b2650abccd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:34.250693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:34.271979 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1043/2083] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:09:34.279693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a183f-e300-44e4-b6cc-9794024c11ab None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:34.281164 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cb6a183f-e300-44e4-b6cc-9794024c11ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:34.296697 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a183f-e300-44e4-b6cc-9794024c11ab None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:34.306479 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf100a-37ce-4969-b38c-ecfbe5b4615c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:34.307210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf100a-37ce-4969-b38c-ecfbe5b4615c service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:34.307510 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf100a-37ce-4969-b38c-ecfbe5b4615c service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:34.307863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf100a-37ce-4969-b38c-ecfbe5b4615c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:34.324635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf100a-37ce-4969-b38c-ecfbe5b4615c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:34.325344 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42592e1-fe27-4d42-b92e-42b2650abccd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:34.326842 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf100a-37ce-4969-b38c-ecfbe5b4615c service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:34.327025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42592e1-fe27-4d42-b92e-42b2650abccd service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:34.337825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1044/2084] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:09:34.342099 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1041/2085] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:34.577055 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf100a-37ce-4969-b38c-ecfbe5b4615c service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:34.586734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf100a-37ce-4969-b38c-ecfbe5b4615c service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:34.590948 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebda4e-c416-4f52-826a-06bb31ed6778 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:34.591900 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebda4e-c416-4f52-826a-06bb31ed6778 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:34.592314 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebda4e-c416-4f52-826a-06bb31ed6778 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:34.592693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebda4e-c416-4f52-826a-06bb31ed6778 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:34.594937 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-51ebda4e-c416-4f52-826a-06bb31ed6778 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:34.685697 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ebda4e-c416-4f52-826a-06bb31ed6778 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:34.687648 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ebda4e-c416-4f52-826a-06bb31ed6778 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:34.703798 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1045/2086] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:35.023552 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42592e1-fe27-4d42-b92e-42b2650abccd service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:35.110871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42592e1-fe27-4d42-b92e-42b2650abccd service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:35.116510 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49351d63-ad67-4dba-b800-1c42e1903664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:35.117523 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49351d63-ad67-4dba-b800-1c42e1903664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:35.117916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49351d63-ad67-4dba-b800-1c42e1903664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:35.118300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49351d63-ad67-4dba-b800-1c42e1903664 admin admin] PATH_INFO: `/v3/users/b0cbdf0853db4442be13457bf905b19c` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:35.119712 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49351d63-ad67-4dba-b800-1c42e1903664 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0cbdf0853db4442be13457bf905b19c)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:35.122126 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49351d63-ad67-4dba-b800-1c42e1903664 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:35.187918 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-49351d63-ad67-4dba-b800-1c42e1903664 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0cbdf0853db4442be13457bf905b19c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:35.196793 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-49351d63-ad67-4dba-b800-1c42e1903664 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0cbdf0853db4442be13457bf905b19c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:35.204320 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-49351d63-ad67-4dba-b800-1c42e1903664 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0cbdf0853db4442be13457bf905b19c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:35.205018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-49351d63-ad67-4dba-b800-1c42e1903664 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0cbdf0853db4442be13457bf905b19c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:35.218959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-49351d63-ad67-4dba-b800-1c42e1903664 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0cbdf0853db4442be13457bf905b19c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:35.225874 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1042/2087] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:35 2022] DELETE /identity/v3/users/b0cbdf0853db4442be13457bf905b19c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:35.230302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda55c9-28bb-4448-b105-73cacae3b8fb None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:35.230751 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda55c9-28bb-4448-b105-73cacae3b8fb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:35.230879 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda55c9-28bb-4448-b105-73cacae3b8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:35.248345 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bbda55c9-28bb-4448-b105-73cacae3b8fb None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:35.249656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bbda55c9-28bb-4448-b105-73cacae3b8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:35.315619 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1046/2088] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:35.325840 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49351d63-ad67-4dba-b800-1c42e1903664 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:35.327495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-49351d63-ad67-4dba-b800-1c42e1903664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:35.342125 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:35.391375 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49351d63-ad67-4dba-b800-1c42e1903664 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:35.395364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac1009-5a45-46d0-bb93-94f08e8267ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:35.396073 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac1009-5a45-46d0-bb93-94f08e8267ee service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:35.396368 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac1009-5a45-46d0-bb93-94f08e8267ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:35.396652 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac1009-5a45-46d0-bb93-94f08e8267ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:35.408460 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac1009-5a45-46d0-bb93-94f08e8267ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:35.409926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac1009-5a45-46d0-bb93-94f08e8267ee service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:35.420569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1043/2089] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:35.858790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda55c9-28bb-4448-b105-73cacae3b8fb None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:35.860407 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bbda55c9-28bb-4448-b105-73cacae3b8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:35.880728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda55c9-28bb-4448-b105-73cacae3b8fb None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:35.890414 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e9b27-6910-462a-adeb-e85d6fba5684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:35.891756 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e9b27-6910-462a-adeb-e85d6fba5684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:35.892616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e9b27-6910-462a-adeb-e85d6fba5684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:35.893914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e9b27-6910-462a-adeb-e85d6fba5684 admin admin] PATH_INFO: `/v3/projects/709e240cf7004f93ad50da59df086482` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:35.896249 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e9b27-6910-462a-adeb-e85d6fba5684 admin admin] RBAC: Authorizing `identity:delete_project(project_id=709e240cf7004f93ad50da59df086482)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:35.898886 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e9b27-6910-462a-adeb-e85d6fba5684 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:35.961440 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-315e9b27-6910-462a-adeb-e85d6fba5684 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '709e240cf7004f93ad50da59df086482', 'request_id': 'req-315e9b27-6910-462a-adeb-e85d6fba5684'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:35.969838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-315e9b27-6910-462a-adeb-e85d6fba5684 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '709e240cf7004f93ad50da59df086482', 'request_id': 'req-315e9b27-6910-462a-adeb-e85d6fba5684'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:35.981241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-315e9b27-6910-462a-adeb-e85d6fba5684 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '709e240cf7004f93ad50da59df086482', 'request_id': 'req-315e9b27-6910-462a-adeb-e85d6fba5684'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:35.990137 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-315e9b27-6910-462a-adeb-e85d6fba5684 admin admin] The token cache is being invalidate because project 709e240cf7004f93ad50da59df086482 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:35.990435 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-315e9b27-6910-462a-adeb-e85d6fba5684 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:35.991691 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-315e9b27-6910-462a-adeb-e85d6fba5684 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:35.993311 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1047/2090] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:35 2022] DELETE /identity/v3/projects/709e240cf7004f93ad50da59df086482 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:35.997428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac1009-5a45-46d0-bb93-94f08e8267ee service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:36.007295 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac1009-5a45-46d0-bb93-94f08e8267ee service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:36.067071 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9bc7a-14fa-4255-82c7-5f5252e2d1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:36.067995 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9bc7a-14fa-4255-82c7-5f5252e2d1bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:36.068291 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9bc7a-14fa-4255-82c7-5f5252e2d1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:36.068669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9bc7a-14fa-4255-82c7-5f5252e2d1bc admin admin] PATH_INFO: `/v3/users/776dc8da9ed84af89b889be47c237955` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:36.069988 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9bc7a-14fa-4255-82c7-5f5252e2d1bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=776dc8da9ed84af89b889be47c237955)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:36.071918 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9bc7a-14fa-4255-82c7-5f5252e2d1bc admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:36.132348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4ac9bc7a-14fa-4255-82c7-5f5252e2d1bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '776dc8da9ed84af89b889be47c237955'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:36.140594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4ac9bc7a-14fa-4255-82c7-5f5252e2d1bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '776dc8da9ed84af89b889be47c237955'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:36.151972 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4ac9bc7a-14fa-4255-82c7-5f5252e2d1bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '776dc8da9ed84af89b889be47c237955'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:36.152695 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4ac9bc7a-14fa-4255-82c7-5f5252e2d1bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '776dc8da9ed84af89b889be47c237955'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:36.166751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4ac9bc7a-14fa-4255-82c7-5f5252e2d1bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '776dc8da9ed84af89b889be47c237955'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:36.173843 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1044/2091] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:35 2022] DELETE /identity/v3/users/776dc8da9ed84af89b889be47c237955 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:36.504176 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e9b27-6910-462a-adeb-e85d6fba5684 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:36.505663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-315e9b27-6910-462a-adeb-e85d6fba5684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:36.516558 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:36.523777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e9b27-6910-462a-adeb-e85d6fba5684 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:36.583820 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc662f-af9a-4235-adde-d4d72641dd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:36.586644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc662f-af9a-4235-adde-d4d72641dd40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:36.586644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc662f-af9a-4235-adde-d4d72641dd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:36.586644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc662f-af9a-4235-adde-d4d72641dd40 admin admin] PATH_INFO: `/v3/projects/81b0cffe45524f19b60da7cf01977e77` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:36.587062 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc662f-af9a-4235-adde-d4d72641dd40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81b0cffe45524f19b60da7cf01977e77)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:36.589089 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc662f-af9a-4235-adde-d4d72641dd40 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:36.634050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7acc662f-af9a-4235-adde-d4d72641dd40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81b0cffe45524f19b60da7cf01977e77', 'request_id': 'req-7acc662f-af9a-4235-adde-d4d72641dd40'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:36.640332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7acc662f-af9a-4235-adde-d4d72641dd40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81b0cffe45524f19b60da7cf01977e77', 'request_id': 'req-7acc662f-af9a-4235-adde-d4d72641dd40'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:36.648779 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7acc662f-af9a-4235-adde-d4d72641dd40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81b0cffe45524f19b60da7cf01977e77', 'request_id': 'req-7acc662f-af9a-4235-adde-d4d72641dd40'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:36.660157 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7acc662f-af9a-4235-adde-d4d72641dd40 admin admin] The token cache is being invalidate because project 81b0cffe45524f19b60da7cf01977e77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:36.660459 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7acc662f-af9a-4235-adde-d4d72641dd40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:36.661115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7acc662f-af9a-4235-adde-d4d72641dd40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:36.662709 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1048/2092] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:36 2022] DELETE /identity/v3/projects/81b0cffe45524f19b60da7cf01977e77 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:36.667633 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422224b-788c-41af-9235-5e1efb514bc8 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:36.667936 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422224b-788c-41af-9235-5e1efb514bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:36.668259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422224b-788c-41af-9235-5e1efb514bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:36.686294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f422224b-788c-41af-9235-5e1efb514bc8 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:36.687500 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f422224b-788c-41af-9235-5e1efb514bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:36.721041 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:36.752110 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1045/2093] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:36.760600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa35509-7dd0-4c82-a27d-0734e63a41ac None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:36.760898 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa35509-7dd0-4c82-a27d-0734e63a41ac None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:36.761185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa35509-7dd0-4c82-a27d-0734e63a41ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:36.779686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4aa35509-7dd0-4c82-a27d-0734e63a41ac None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:36.781068 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4aa35509-7dd0-4c82-a27d-0734e63a41ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:36.789558 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1049/2094] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:36.793674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422224b-788c-41af-9235-5e1efb514bc8 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:36.803612 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422224b-788c-41af-9235-5e1efb514bc8 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:36.808354 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade572f-a01d-450a-a1bd-0069be540d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:36.809967 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade572f-a01d-450a-a1bd-0069be540d25 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:36.809967 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade572f-a01d-450a-a1bd-0069be540d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:36.809967 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade572f-a01d-450a-a1bd-0069be540d25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:36.810424 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade572f-a01d-450a-a1bd-0069be540d25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:36.810611 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade572f-a01d-450a-a1bd-0069be540d25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:36.812375 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade572f-a01d-450a-a1bd-0069be540d25 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:36.828785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1046/2095] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:09:36.833369 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa35509-7dd0-4c82-a27d-0734e63a41ac None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:36.844667 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa35509-7dd0-4c82-a27d-0734e63a41ac None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:36.852503 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e0f2b-b925-45fd-9776-67c949899c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:36.853293 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e0f2b-b925-45fd-9776-67c949899c3f admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:36.853683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e0f2b-b925-45fd-9776-67c949899c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:36.854056 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e0f2b-b925-45fd-9776-67c949899c3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:36.854844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e0f2b-b925-45fd-9776-67c949899c3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:36.856887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e0f2b-b925-45fd-9776-67c949899c3f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:36.886334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1050/2096] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:36 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:36.890449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade572f-a01d-450a-a1bd-0069be540d25 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:36.902329 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade572f-a01d-450a-a1bd-0069be540d25 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:36.971016 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eea631-0c76-4623-a750-4608a0d73f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:36.971701 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eea631-0c76-4623-a750-4608a0d73f7b admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:36.972029 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eea631-0c76-4623-a750-4608a0d73f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:36.972728 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eea631-0c76-4623-a750-4608a0d73f7b admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:36.973702 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eea631-0c76-4623-a750-4608a0d73f7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:36.975106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eea631-0c76-4623-a750-4608a0d73f7b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:37.008036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1047/2097] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:36 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:09:37.011931 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e0f2b-b925-45fd-9776-67c949899c3f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:37.027964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e0f2b-b925-45fd-9776-67c949899c3f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:37.033834 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25e0e1-38e3-4efa-84e1-2cb6cf6b20bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:37.034966 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25e0e1-38e3-4efa-84e1-2cb6cf6b20bb admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:37.035540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25e0e1-38e3-4efa-84e1-2cb6cf6b20bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:37.036015 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25e0e1-38e3-4efa-84e1-2cb6cf6b20bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:37.036958 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25e0e1-38e3-4efa-84e1-2cb6cf6b20bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:37.039430 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25e0e1-38e3-4efa-84e1-2cb6cf6b20bb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:37.056758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1051/2098] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:37.060574 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eea631-0c76-4623-a750-4608a0d73f7b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:37.077318 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eea631-0c76-4623-a750-4608a0d73f7b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:37.083646 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4bcdb7-73f9-467d-b210-f9a2d854717a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:37.084856 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4bcdb7-73f9-467d-b210-f9a2d854717a admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:37.085230 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4bcdb7-73f9-467d-b210-f9a2d854717a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:37.085594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4bcdb7-73f9-467d-b210-f9a2d854717a admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:37.086410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4bcdb7-73f9-467d-b210-f9a2d854717a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:37.088147 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4bcdb7-73f9-467d-b210-f9a2d854717a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:37.102550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1048/2099] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:37.103979 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25e0e1-38e3-4efa-84e1-2cb6cf6b20bb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:37.114630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25e0e1-38e3-4efa-84e1-2cb6cf6b20bb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:37.120079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38af86-c5d1-4046-8381-d90456cc23a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:37.121049 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38af86-c5d1-4046-8381-d90456cc23a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:37.121442 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38af86-c5d1-4046-8381-d90456cc23a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:37.121786 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38af86-c5d1-4046-8381-d90456cc23a1 admin admin] PATH_INFO: `/v3/projects/fe97437d94f249f1830c92d98832f5ff/users/d41d7740b55e42ffa3e7a015c41bdbf9/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:37.137402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b38af86-c5d1-4046-8381-d90456cc23a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe97437d94f249f1830c92d98832f5ff, user_id=d41d7740b55e42ffa3e7a015c41bdbf9, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:37.140541 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b38af86-c5d1-4046-8381-d90456cc23a1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:37.157615 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1052/2100] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:37 2022] PUT /identity/v3/projects/fe97437d94f249f1830c92d98832f5ff/users/d41d7740b55e42ffa3e7a015c41bdbf9/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:37.167778 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc4714-3986-41f7-9da6-ad38dd0ad9b4 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:37.168307 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc4714-3986-41f7-9da6-ad38dd0ad9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:37.168428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc4714-3986-41f7-9da6-ad38dd0ad9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:37.198486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b8bc4714-3986-41f7-9da6-ad38dd0ad9b4 None None] MFA Rules not processed for user `d41d7740b55e42ffa3e7a015c41bdbf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:37.199803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b8bc4714-3986-41f7-9da6-ad38dd0ad9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:37.268582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1049/2101] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:37.273180 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38af86-c5d1-4046-8381-d90456cc23a1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:37.307185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:37.342457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38af86-c5d1-4046-8381-d90456cc23a1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:37.347192 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a00bd-5d0d-4123-a2c5-d99c666aebca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:37.347859 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a00bd-5d0d-4123-a2c5-d99c666aebca admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:37.348183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a00bd-5d0d-4123-a2c5-d99c666aebca admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:37.348468 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a00bd-5d0d-4123-a2c5-d99c666aebca admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:37.349178 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a00bd-5d0d-4123-a2c5-d99c666aebca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:37.354351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a00bd-5d0d-4123-a2c5-d99c666aebca admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:37.381571 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1053/2102] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:37 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:09:37.385526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc4714-3986-41f7-9da6-ad38dd0ad9b4 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:37.398867 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc4714-3986-41f7-9da6-ad38dd0ad9b4 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:37.431386 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:37.461866 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f64c653-d7dd-42ac-8e8d-2d961b8ee9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:37.462631 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64c653-d7dd-42ac-8e8d-2d961b8ee9c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:37.462996 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64c653-d7dd-42ac-8e8d-2d961b8ee9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:37.463261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64c653-d7dd-42ac-8e8d-2d961b8ee9c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:37.464141 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f64c653-d7dd-42ac-8e8d-2d961b8ee9c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:37.466142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f64c653-d7dd-42ac-8e8d-2d961b8ee9c2 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:37.488779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1050/2103] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:37 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:37.490013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a00bd-5d0d-4123-a2c5-d99c666aebca admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:37.500484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a00bd-5d0d-4123-a2c5-d99c666aebca admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:37.505408 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49849468-cfff-4e81-9665-84e6ce8d0724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:37.506323 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49849468-cfff-4e81-9665-84e6ce8d0724 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:37.507266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49849468-cfff-4e81-9665-84e6ce8d0724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:37.507689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49849468-cfff-4e81-9665-84e6ce8d0724 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:37.508580 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49849468-cfff-4e81-9665-84e6ce8d0724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:37.510444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49849468-cfff-4e81-9665-84e6ce8d0724 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:37.521916 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1054/2104] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:37.526541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f64c653-d7dd-42ac-8e8d-2d961b8ee9c2 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:37.536545 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f64c653-d7dd-42ac-8e8d-2d961b8ee9c2 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:37.541293 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8b46d-8690-4b6d-b479-d91052c9fa4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:37.542169 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8b46d-8690-4b6d-b479-d91052c9fa4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:37.542460 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8b46d-8690-4b6d-b479-d91052c9fa4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:37.542746 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8b46d-8690-4b6d-b479-d91052c9fa4e admin admin] PATH_INFO: `/v3/projects/394863796f5749aaafa760342b3bc1c0/users/aa7edbcedf594068ab006e90e0e154a3/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:37.558586 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8b46d-8690-4b6d-b479-d91052c9fa4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=394863796f5749aaafa760342b3bc1c0, user_id=aa7edbcedf594068ab006e90e0e154a3, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:37.561309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8b46d-8690-4b6d-b479-d91052c9fa4e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:37.576386 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1051/2105] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:37 2022] PUT /identity/v3/projects/394863796f5749aaafa760342b3bc1c0/users/aa7edbcedf594068ab006e90e0e154a3/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:37.580310 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49849468-cfff-4e81-9665-84e6ce8d0724 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:37.591654 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49849468-cfff-4e81-9665-84e6ce8d0724 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:37.670472 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288cc627-e840-4a46-b045-b22d7b163d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:37.671059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cc627-e840-4a46-b045-b22d7b163d3a admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:37.671352 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cc627-e840-4a46-b045-b22d7b163d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:37.671635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cc627-e840-4a46-b045-b22d7b163d3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:37.672497 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288cc627-e840-4a46-b045-b22d7b163d3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:37.678481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288cc627-e840-4a46-b045-b22d7b163d3a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:37.697775 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1055/2106] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:37.701975 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8b46d-8690-4b6d-b479-d91052c9fa4e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:37.713414 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8b46d-8690-4b6d-b479-d91052c9fa4e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:37.718067 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0cb9b-0623-406b-b301-d4200a262bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:37.718943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0cb9b-0623-406b-b301-d4200a262bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:37.719228 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0cb9b-0623-406b-b301-d4200a262bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:37.719504 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0cb9b-0623-406b-b301-d4200a262bdc admin admin] PATH_INFO: `/v3/projects/394863796f5749aaafa760342b3bc1c0/users/aa7edbcedf594068ab006e90e0e154a3/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:37.721484 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0cb9b-0623-406b-b301-d4200a262bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=394863796f5749aaafa760342b3bc1c0, user_id=aa7edbcedf594068ab006e90e0e154a3, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:37.724121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0cb9b-0623-406b-b301-d4200a262bdc admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:37.734243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1052/2107] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:37 2022] PUT /identity/v3/projects/394863796f5749aaafa760342b3bc1c0/users/aa7edbcedf594068ab006e90e0e154a3/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:37.801420 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb3e60-3408-40b3-9589-eaa7441bfc01 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:37.801747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb3e60-3408-40b3-9589-eaa7441bfc01 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:37.802036 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb3e60-3408-40b3-9589-eaa7441bfc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:37.829426 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9bbb3e60-3408-40b3-9589-eaa7441bfc01 None None] MFA Rules not processed for user `aa7edbcedf594068ab006e90e0e154a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:37.830660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9bbb3e60-3408-40b3-9589-eaa7441bfc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:37.896552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1056/2108] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:37 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:37.906961 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0cb9b-0623-406b-b301-d4200a262bdc admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:37.908668 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-38d0cb9b-0623-406b-b301-d4200a262bdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:37.965584 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0cb9b-0623-406b-b301-d4200a262bdc admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:37.969228 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5d819-74d8-43b3-a1b9-412bde629b5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:37.969878 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5d819-74d8-43b3-a1b9-412bde629b5e service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:37.970168 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5d819-74d8-43b3-a1b9-412bde629b5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:37.970451 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5d819-74d8-43b3-a1b9-412bde629b5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:38.030909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5d819-74d8-43b3-a1b9-412bde629b5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:38.032386 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5d819-74d8-43b3-a1b9-412bde629b5e service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:38.041819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1053/2109] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:38.113349 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb3e60-3408-40b3-9589-eaa7441bfc01 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:38.114764 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9bbb3e60-3408-40b3-9589-eaa7441bfc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:38.143348 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:38.183347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb3e60-3408-40b3-9589-eaa7441bfc01 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:38.187862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334c546-b7b6-4dd8-9672-f5569c24ce24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:38.188767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334c546-b7b6-4dd8-9672-f5569c24ce24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:38.189054 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334c546-b7b6-4dd8-9672-f5569c24ce24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:38.189335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334c546-b7b6-4dd8-9672-f5569c24ce24 admin admin] PATH_INFO: `/v3/users/a30ecfe7bbe9476cae379d9c08db318d` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:38.190827 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334c546-b7b6-4dd8-9672-f5569c24ce24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a30ecfe7bbe9476cae379d9c08db318d)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:38.192770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334c546-b7b6-4dd8-9672-f5569c24ce24 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:38.239659 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d334c546-b7b6-4dd8-9672-f5569c24ce24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a30ecfe7bbe9476cae379d9c08db318d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:38.249382 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d334c546-b7b6-4dd8-9672-f5569c24ce24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a30ecfe7bbe9476cae379d9c08db318d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:38.257450 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d334c546-b7b6-4dd8-9672-f5569c24ce24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a30ecfe7bbe9476cae379d9c08db318d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:38.258199 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d334c546-b7b6-4dd8-9672-f5569c24ce24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a30ecfe7bbe9476cae379d9c08db318d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:38.271719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d334c546-b7b6-4dd8-9672-f5569c24ce24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a30ecfe7bbe9476cae379d9c08db318d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:38.278380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1057/2110] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:38 2022] DELETE /identity/v3/users/a30ecfe7bbe9476cae379d9c08db318d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:38.282986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502e8fc-96ca-4c00-9dad-80bdabedcb88 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:38.283785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502e8fc-96ca-4c00-9dad-80bdabedcb88 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:38.284113 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502e8fc-96ca-4c00-9dad-80bdabedcb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:38.304984 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-e502e8fc-96ca-4c00-9dad-80bdabedcb88 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:38.306431 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e502e8fc-96ca-4c00-9dad-80bdabedcb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:38.375234 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:38.427977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1054/2111] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:09:38.445840 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334c546-b7b6-4dd8-9672-f5569c24ce24 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:38.448463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d334c546-b7b6-4dd8-9672-f5569c24ce24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:38.524204 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334c546-b7b6-4dd8-9672-f5569c24ce24 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:38.531144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc245cb-4aec-4781-8d92-2315ebff83dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:38.532064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc245cb-4aec-4781-8d92-2315ebff83dd service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:38.532516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc245cb-4aec-4781-8d92-2315ebff83dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:38.532900 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc245cb-4aec-4781-8d92-2315ebff83dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:38.543901 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc245cb-4aec-4781-8d92-2315ebff83dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:38.545498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc245cb-4aec-4781-8d92-2315ebff83dd service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:38.556332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1058/2112] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:38.829115 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502e8fc-96ca-4c00-9dad-80bdabedcb88 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:38.830597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e502e8fc-96ca-4c00-9dad-80bdabedcb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:38.843283 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502e8fc-96ca-4c00-9dad-80bdabedcb88 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:38.848146 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82b599-9d6c-46e4-b632-452af6fbf9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:38.849008 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82b599-9d6c-46e4-b632-452af6fbf9b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:38.849298 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82b599-9d6c-46e4-b632-452af6fbf9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:38.849581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82b599-9d6c-46e4-b632-452af6fbf9b9 admin admin] PATH_INFO: `/v3/projects/7312a1e2b25b4a718747bc99c98d64fe` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:38.850809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82b599-9d6c-46e4-b632-452af6fbf9b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7312a1e2b25b4a718747bc99c98d64fe)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:38.852766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82b599-9d6c-46e4-b632-452af6fbf9b9 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:38.899404 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5d82b599-9d6c-46e4-b632-452af6fbf9b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7312a1e2b25b4a718747bc99c98d64fe', 'request_id': 'req-5d82b599-9d6c-46e4-b632-452af6fbf9b9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:38.904934 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5d82b599-9d6c-46e4-b632-452af6fbf9b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7312a1e2b25b4a718747bc99c98d64fe', 'request_id': 'req-5d82b599-9d6c-46e4-b632-452af6fbf9b9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:38.912306 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5d82b599-9d6c-46e4-b632-452af6fbf9b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7312a1e2b25b4a718747bc99c98d64fe', 'request_id': 'req-5d82b599-9d6c-46e4-b632-452af6fbf9b9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:38.923309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5d82b599-9d6c-46e4-b632-452af6fbf9b9 admin admin] The token cache is being invalidate because project 7312a1e2b25b4a718747bc99c98d64fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:38.923602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5d82b599-9d6c-46e4-b632-452af6fbf9b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:38.924245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5d82b599-9d6c-46e4-b632-452af6fbf9b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:38.925780 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1055/2113] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:38 2022] DELETE /identity/v3/projects/7312a1e2b25b4a718747bc99c98d64fe => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:38.930638 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d79b4-a7f7-4f0b-be36-3e5bf45be1e0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:38.930937 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d79b4-a7f7-4f0b-be36-3e5bf45be1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:38.931221 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d79b4-a7f7-4f0b-be36-3e5bf45be1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:38.949379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5a2d79b4-a7f7-4f0b-be36-3e5bf45be1e0 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:38.950596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5a2d79b4-a7f7-4f0b-be36-3e5bf45be1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:38.991867 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:39.011262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1059/2114] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:39.018967 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a3d85-1699-4a2e-9a10-1d75f02596d2 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:39.019264 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a3d85-1699-4a2e-9a10-1d75f02596d2 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:39.019554 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a3d85-1699-4a2e-9a10-1d75f02596d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:39.046759 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-651a3d85-1699-4a2e-9a10-1d75f02596d2 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:39.047998 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-651a3d85-1699-4a2e-9a10-1d75f02596d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:39.055833 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1056/2115] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:39.060087 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d79b4-a7f7-4f0b-be36-3e5bf45be1e0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:39.070634 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d79b4-a7f7-4f0b-be36-3e5bf45be1e0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:39.075183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c9daa-561f-4b22-bd5f-671afd68b995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:39.075831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c9daa-561f-4b22-bd5f-671afd68b995 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:39.076148 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c9daa-561f-4b22-bd5f-671afd68b995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:39.076436 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c9daa-561f-4b22-bd5f-671afd68b995 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:39.077166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c9daa-561f-4b22-bd5f-671afd68b995 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:39.077482 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c9daa-561f-4b22-bd5f-671afd68b995 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:39.079111 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c9daa-561f-4b22-bd5f-671afd68b995 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:39.091169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1060/2116] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:39.095208 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a3d85-1699-4a2e-9a10-1d75f02596d2 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:39.104439 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a3d85-1699-4a2e-9a10-1d75f02596d2 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:39.109067 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c92dd0-54bb-4ea5-aafc-9d93002dbd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:39.109741 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c92dd0-54bb-4ea5-aafc-9d93002dbd99 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:39.110029 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c92dd0-54bb-4ea5-aafc-9d93002dbd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:39.110308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c92dd0-54bb-4ea5-aafc-9d93002dbd99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:39.110994 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c92dd0-54bb-4ea5-aafc-9d93002dbd99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:39.112929 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c92dd0-54bb-4ea5-aafc-9d93002dbd99 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:39.138698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1057/2117] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:39 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:39.150249 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c9daa-561f-4b22-bd5f-671afd68b995 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:39.160642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c9daa-561f-4b22-bd5f-671afd68b995 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:39.217422 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a9518-7f95-4ffd-bdf2-c36330d977cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:39.218150 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a9518-7f95-4ffd-bdf2-c36330d977cf admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:39.218445 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a9518-7f95-4ffd-bdf2-c36330d977cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:39.218731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a9518-7f95-4ffd-bdf2-c36330d977cf admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:39.219475 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a9518-7f95-4ffd-bdf2-c36330d977cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:39.221488 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a9518-7f95-4ffd-bdf2-c36330d977cf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:39.242944 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1061/2118] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:39 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:39.247027 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c92dd0-54bb-4ea5-aafc-9d93002dbd99 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:39.258540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c92dd0-54bb-4ea5-aafc-9d93002dbd99 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:39.263547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc24b40-6284-4321-8372-001ebb6d2068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:39.264270 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc24b40-6284-4321-8372-001ebb6d2068 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:39.264582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc24b40-6284-4321-8372-001ebb6d2068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:39.264861 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc24b40-6284-4321-8372-001ebb6d2068 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:39.265625 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc24b40-6284-4321-8372-001ebb6d2068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:39.267229 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc24b40-6284-4321-8372-001ebb6d2068 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:39.278181 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1058/2119] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:39.282411 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a9518-7f95-4ffd-bdf2-c36330d977cf admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:39.292447 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a9518-7f95-4ffd-bdf2-c36330d977cf admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:39.297051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ffce1c-17f1-4068-8b20-471164405aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:39.297705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffce1c-17f1-4068-8b20-471164405aaa admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:39.297991 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffce1c-17f1-4068-8b20-471164405aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:39.298268 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffce1c-17f1-4068-8b20-471164405aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:39.298988 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ffce1c-17f1-4068-8b20-471164405aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:39.300639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ffce1c-17f1-4068-8b20-471164405aaa admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:39.311103 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1062/2120] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:39.315739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc24b40-6284-4321-8372-001ebb6d2068 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:39.326627 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc24b40-6284-4321-8372-001ebb6d2068 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:39.331278 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124a3cb-1747-4961-977e-14bab9af73c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:39.332199 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124a3cb-1747-4961-977e-14bab9af73c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:39.332487 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124a3cb-1747-4961-977e-14bab9af73c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:39.332790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124a3cb-1747-4961-977e-14bab9af73c2 admin admin] PATH_INFO: `/v3/projects/9f6e581c160d4e64a81e0b35907cc078/users/da8ecff90e714e4e961f8a8104143f62/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:39.348441 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124a3cb-1747-4961-977e-14bab9af73c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f6e581c160d4e64a81e0b35907cc078, user_id=da8ecff90e714e4e961f8a8104143f62, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:39.351713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124a3cb-1747-4961-977e-14bab9af73c2 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:39.366380 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1059/2121] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:39 2022] PUT /identity/v3/projects/9f6e581c160d4e64a81e0b35907cc078/users/da8ecff90e714e4e961f8a8104143f62/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:39.371025 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0f07d-5d09-4193-8e29-f324de890e14 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:39.371355 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0f07d-5d09-4193-8e29-f324de890e14 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:39.371660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0f07d-5d09-4193-8e29-f324de890e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:39.392953 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-14c0f07d-5d09-4193-8e29-f324de890e14 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:39.394939 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-14c0f07d-5d09-4193-8e29-f324de890e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:39.468483 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1063/2122] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:39.479378 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124a3cb-1747-4961-977e-14bab9af73c2 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:39.481269 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8124a3cb-1747-4961-977e-14bab9af73c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:39.557495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124a3cb-1747-4961-977e-14bab9af73c2 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:39.562691 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918bc25-f0e9-474b-a9bd-93dd7c7e3598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:39.563595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918bc25-f0e9-474b-a9bd-93dd7c7e3598 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:39.564008 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918bc25-f0e9-474b-a9bd-93dd7c7e3598 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:39.564394 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918bc25-f0e9-474b-a9bd-93dd7c7e3598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:39.575542 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918bc25-f0e9-474b-a9bd-93dd7c7e3598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:39.577178 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918bc25-f0e9-474b-a9bd-93dd7c7e3598 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:39.583527 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:39.590494 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1060/2123] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:40.508857 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96eae00-5e7b-4131-8a4f-62a01dcef593 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:40.509398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96eae00-5e7b-4131-8a4f-62a01dcef593 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:40.509398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96eae00-5e7b-4131-8a4f-62a01dcef593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:40.539781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f96eae00-5e7b-4131-8a4f-62a01dcef593 None None] MFA Rules not processed for user `da8ecff90e714e4e961f8a8104143f62`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:40.541059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f96eae00-5e7b-4131-8a4f-62a01dcef593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:40.589290 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:40.625318 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1064/2124] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:40.633582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918bc25-f0e9-474b-a9bd-93dd7c7e3598 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:40.636078 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8918bc25-f0e9-474b-a9bd-93dd7c7e3598 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:40.696814 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918bc25-f0e9-474b-a9bd-93dd7c7e3598 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:40.701751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe36c30-94db-4aa8-9b24-1a1b5311d415 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:40.702577 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe36c30-94db-4aa8-9b24-1a1b5311d415 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:40.702963 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe36c30-94db-4aa8-9b24-1a1b5311d415 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:40.703473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe36c30-94db-4aa8-9b24-1a1b5311d415 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:40.713866 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe36c30-94db-4aa8-9b24-1a1b5311d415 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:40.715323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe36c30-94db-4aa8-9b24-1a1b5311d415 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:40.724778 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1061/2125] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:43.245569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3511c-c66d-4d30-a9f2-e91ff2d9db68 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:43.246004 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3511c-c66d-4d30-a9f2-e91ff2d9db68 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:43.246139 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3511c-c66d-4d30-a9f2-e91ff2d9db68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:43.264215 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-edd3511c-c66d-4d30-a9f2-e91ff2d9db68 None None] MFA Rules not processed for user `da8ecff90e714e4e961f8a8104143f62`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:43.265424 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-edd3511c-c66d-4d30-a9f2-e91ff2d9db68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:43.276361 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1065/2126] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:44.708696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe36c30-94db-4aa8-9b24-1a1b5311d415 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:44.711201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0fe36c30-94db-4aa8-9b24-1a1b5311d415 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:44.728266 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe36c30-94db-4aa8-9b24-1a1b5311d415 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:44.732076 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db716e-cef7-4391-8984-d129e588ff58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:44.732730 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db716e-cef7-4391-8984-d129e588ff58 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:44.733022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db716e-cef7-4391-8984-d129e588ff58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:44.733306 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db716e-cef7-4391-8984-d129e588ff58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:44.736523 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-46db716e-cef7-4391-8984-d129e588ff58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:44.812179 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db716e-cef7-4391-8984-d129e588ff58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:44.813845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db716e-cef7-4391-8984-d129e588ff58 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:44.830942 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1062/2127] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:48.353829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d514661-e99e-4fb4-b1eb-229bdf921813 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:48.354970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d514661-e99e-4fb4-b1eb-229bdf921813 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:48.355408 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d514661-e99e-4fb4-b1eb-229bdf921813 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:48.360018 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1066/2128] 10.222.0.94 () {54 vars in 921 bytes} [Mon Jul 18 14:09:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:09:48.399206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa6f67-8759-4b9e-a3f5-625aeb194f13 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:48.399648 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa6f67-8759-4b9e-a3f5-625aeb194f13 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:48.399779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa6f67-8759-4b9e-a3f5-625aeb194f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:48.421034 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-0aaa6f67-8759-4b9e-a3f5-625aeb194f13 None None] MFA Rules not processed for user `dd5b2ba9e37041c78a99f1700a4970c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:48.422278 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0aaa6f67-8759-4b9e-a3f5-625aeb194f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:48.427737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d514661-e99e-4fb4-b1eb-229bdf921813 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:48.429424 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5d514661-e99e-4fb4-b1eb-229bdf921813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:48.448623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d514661-e99e-4fb4-b1eb-229bdf921813 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:48.453357 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce899be-b7c1-4eca-8fef-7eec0e44e0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:48.454227 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce899be-b7c1-4eca-8fef-7eec0e44e0e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:48.454513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce899be-b7c1-4eca-8fef-7eec0e44e0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:48.454805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce899be-b7c1-4eca-8fef-7eec0e44e0e7 admin admin] PATH_INFO: `/v3/users/226d34540e464a93aa984ff59b497450` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:48.456026 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce899be-b7c1-4eca-8fef-7eec0e44e0e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=226d34540e464a93aa984ff59b497450)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:48.457974 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce899be-b7c1-4eca-8fef-7eec0e44e0e7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:48.493957 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1063/2129] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Jul 18 14:09:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:09:48.505168 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa6f67-8759-4b9e-a3f5-625aeb194f13 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:48.506716 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0aaa6f67-8759-4b9e-a3f5-625aeb194f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:48.515824 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1ce899be-b7c1-4eca-8fef-7eec0e44e0e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '226d34540e464a93aa984ff59b497450'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:48.522995 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa6f67-8759-4b9e-a3f5-625aeb194f13 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:48.524908 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1ce899be-b7c1-4eca-8fef-7eec0e44e0e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '226d34540e464a93aa984ff59b497450'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:48.529045 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1feb0a-ebd0-43d6-a003-c7e64b1d6a27 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd5b2ba9e37041c78a99f1700a4970c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:48.529985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1feb0a-ebd0-43d6-a003-c7e64b1d6a27 service placement] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:48.530521 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1feb0a-ebd0-43d6-a003-c7e64b1d6a27 service placement] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:48.530881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1feb0a-ebd0-43d6-a003-c7e64b1d6a27 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:48.537114 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1ce899be-b7c1-4eca-8fef-7eec0e44e0e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '226d34540e464a93aa984ff59b497450'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:48.539367 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1ce899be-b7c1-4eca-8fef-7eec0e44e0e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '226d34540e464a93aa984ff59b497450'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:48.541306 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1feb0a-ebd0-43d6-a003-c7e64b1d6a27 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:48.542975 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1feb0a-ebd0-43d6-a003-c7e64b1d6a27 service placement] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:48.553909 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1ce899be-b7c1-4eca-8fef-7eec0e44e0e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '226d34540e464a93aa984ff59b497450'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:48.558576 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1064/2130] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:09:48.559308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1feb0a-ebd0-43d6-a003-c7e64b1d6a27 service placement] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:48.560724 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4d1feb0a-ebd0-43d6-a003-c7e64b1d6a27 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:48.567361 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1067/2131] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:48 2022] DELETE /identity/v3/users/226d34540e464a93aa984ff59b497450 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:48.579578 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1feb0a-ebd0-43d6-a003-c7e64b1d6a27 service placement] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:48.608232 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:48.663924 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd8c81-cfee-4c93-8a95-643d7a3faa27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:48.664995 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd8c81-cfee-4c93-8a95-643d7a3faa27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:48.665361 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd8c81-cfee-4c93-8a95-643d7a3faa27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:48.665711 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd8c81-cfee-4c93-8a95-643d7a3faa27 admin admin] PATH_INFO: `/v3/users/a197a1588b94405d90bbb9cb45ed855b` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:48.667004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd8c81-cfee-4c93-8a95-643d7a3faa27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a197a1588b94405d90bbb9cb45ed855b)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:48.669104 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd8c81-cfee-4c93-8a95-643d7a3faa27 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:48.734915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0bbd8c81-cfee-4c93-8a95-643d7a3faa27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a197a1588b94405d90bbb9cb45ed855b'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:48.749209 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0bbd8c81-cfee-4c93-8a95-643d7a3faa27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a197a1588b94405d90bbb9cb45ed855b'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:48.757331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0bbd8c81-cfee-4c93-8a95-643d7a3faa27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a197a1588b94405d90bbb9cb45ed855b'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:48.759799 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0bbd8c81-cfee-4c93-8a95-643d7a3faa27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a197a1588b94405d90bbb9cb45ed855b'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:48.777740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0bbd8c81-cfee-4c93-8a95-643d7a3faa27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a197a1588b94405d90bbb9cb45ed855b'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:48.780557 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce899be-b7c1-4eca-8fef-7eec0e44e0e7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:48.782734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1ce899be-b7c1-4eca-8fef-7eec0e44e0e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:48.788586 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1065/2132] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:48 2022] DELETE /identity/v3/users/a197a1588b94405d90bbb9cb45ed855b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:48.794941 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4582b3-571a-48ba-b96d-cf3fedab0f08 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:48.795244 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4582b3-571a-48ba-b96d-cf3fedab0f08 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:48.795532 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4582b3-571a-48ba-b96d-cf3fedab0f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:48.815238 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce899be-b7c1-4eca-8fef-7eec0e44e0e7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:48.821378 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-8b4582b3-571a-48ba-b96d-cf3fedab0f08 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:48.822770 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8b4582b3-571a-48ba-b96d-cf3fedab0f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:48.882530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418e21b-f19c-4975-b228-199744cd4065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:48.883415 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418e21b-f19c-4975-b228-199744cd4065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:48.883702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418e21b-f19c-4975-b228-199744cd4065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:48.884013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418e21b-f19c-4975-b228-199744cd4065 admin admin] PATH_INFO: `/v3/projects/799dde59367144af9f1f8cce1cb7d435` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:48.885219 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418e21b-f19c-4975-b228-199744cd4065 admin admin] RBAC: Authorizing `identity:delete_project(project_id=799dde59367144af9f1f8cce1cb7d435)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:48.887195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418e21b-f19c-4975-b228-199744cd4065 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:48.894503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1066/2133] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:48.906318 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4582b3-571a-48ba-b96d-cf3fedab0f08 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:48.908088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8b4582b3-571a-48ba-b96d-cf3fedab0f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:48.959883 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c418e21b-f19c-4975-b228-199744cd4065 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '799dde59367144af9f1f8cce1cb7d435', 'request_id': 'req-c418e21b-f19c-4975-b228-199744cd4065'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:48.961749 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:48.962152 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:48.972181 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c418e21b-f19c-4975-b228-199744cd4065 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '799dde59367144af9f1f8cce1cb7d435', 'request_id': 'req-c418e21b-f19c-4975-b228-199744cd4065'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:48.981497 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4582b3-571a-48ba-b96d-cf3fedab0f08 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:48.982648 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c418e21b-f19c-4975-b228-199744cd4065 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '799dde59367144af9f1f8cce1cb7d435', 'request_id': 'req-c418e21b-f19c-4975-b228-199744cd4065'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:48.996521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c418e21b-f19c-4975-b228-199744cd4065 admin admin] The token cache is being invalidate because project 799dde59367144af9f1f8cce1cb7d435 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:48.996995 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c418e21b-f19c-4975-b228-199744cd4065 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:48.997727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c418e21b-f19c-4975-b228-199744cd4065 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:48.999660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1068/2134] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:48 2022] DELETE /identity/v3/projects/799dde59367144af9f1f8cce1cb7d435 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:49.007228 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143d198-e54c-4a92-a9f4-fb5e9203169a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:49.007541 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143d198-e54c-4a92-a9f4-fb5e9203169a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:49.007838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143d198-e54c-4a92-a9f4-fb5e9203169a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:49.031139 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4143d198-e54c-4a92-a9f4-fb5e9203169a None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:49.032419 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4143d198-e54c-4a92-a9f4-fb5e9203169a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:49.042048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e12bb9-167f-4d75-bf42-574ca090d975 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:49.042923 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e12bb9-167f-4d75-bf42-574ca090d975 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:49.043373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e12bb9-167f-4d75-bf42-574ca090d975 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:49.043820 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e12bb9-167f-4d75-bf42-574ca090d975 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:49.046019 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b6e12bb9-167f-4d75-bf42-574ca090d975 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:49.094800 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1069/2135] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:09:49.103930 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e119d0f-774d-4932-91ea-6d77974a3470 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:49.104428 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e119d0f-774d-4932-91ea-6d77974a3470 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:49.104818 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e119d0f-774d-4932-91ea-6d77974a3470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:49.111221 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e12bb9-167f-4d75-bf42-574ca090d975 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:49.112705 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e12bb9-167f-4d75-bf42-574ca090d975 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:49.123143 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1067/2136] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:09:49.124980 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7e119d0f-774d-4932-91ea-6d77974a3470 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:49.126233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7e119d0f-774d-4932-91ea-6d77974a3470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:49.134753 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1070/2137] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:49.139083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e12bb9-167f-4d75-bf42-574ca090d975 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:49.149188 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e12bb9-167f-4d75-bf42-574ca090d975 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:49.153841 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3eef7f-0f65-4983-b3b8-b5b0fad28d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:49.154498 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3eef7f-0f65-4983-b3b8-b5b0fad28d01 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:49.154789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3eef7f-0f65-4983-b3b8-b5b0fad28d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:49.155068 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3eef7f-0f65-4983-b3b8-b5b0fad28d01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:49.155828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3eef7f-0f65-4983-b3b8-b5b0fad28d01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:49.156171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3eef7f-0f65-4983-b3b8-b5b0fad28d01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:49.157771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3eef7f-0f65-4983-b3b8-b5b0fad28d01 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:49.170309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1068/2138] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:09:49.174810 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e119d0f-774d-4932-91ea-6d77974a3470 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:49.184928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e119d0f-774d-4932-91ea-6d77974a3470 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:49.190448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0776c12-6997-453e-9c4e-edbb4623e437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:49.191106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776c12-6997-453e-9c4e-edbb4623e437 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:49.191423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776c12-6997-453e-9c4e-edbb4623e437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:49.191708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0776c12-6997-453e-9c4e-edbb4623e437 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:49.192542 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0776c12-6997-453e-9c4e-edbb4623e437 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:49.194975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0776c12-6997-453e-9c4e-edbb4623e437 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:49.223048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1071/2139] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:49 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:49.227491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3eef7f-0f65-4983-b3b8-b5b0fad28d01 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:49.238068 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3eef7f-0f65-4983-b3b8-b5b0fad28d01 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:49.315275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e36663-c021-4157-b895-55728038bf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:49.316055 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e36663-c021-4157-b895-55728038bf05 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:49.317197 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e36663-c021-4157-b895-55728038bf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:49.317197 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e36663-c021-4157-b895-55728038bf05 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:49.317390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e36663-c021-4157-b895-55728038bf05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:49.319455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e36663-c021-4157-b895-55728038bf05 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:49.341639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1069/2140] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:49 2022] POST /identity/v3/users => generated 285 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:49.345923 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0776c12-6997-453e-9c4e-edbb4623e437 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:49.357422 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0776c12-6997-453e-9c4e-edbb4623e437 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:49.363980 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a7727-fdc5-4139-94ad-d72423226585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:49.364669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a7727-fdc5-4139-94ad-d72423226585 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:49.364965 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a7727-fdc5-4139-94ad-d72423226585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:49.365324 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a7727-fdc5-4139-94ad-d72423226585 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:49.366559 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a7727-fdc5-4139-94ad-d72423226585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:49.368210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a7727-fdc5-4139-94ad-d72423226585 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:49.382256 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1072/2141] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:49.386790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e36663-c021-4157-b895-55728038bf05 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:49.399174 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e36663-c021-4157-b895-55728038bf05 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:49.404515 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8865e-0c7b-4b88-95f9-738ebdbe5539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:49.405195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8865e-0c7b-4b88-95f9-738ebdbe5539 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:49.405484 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8865e-0c7b-4b88-95f9-738ebdbe5539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:49.405771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8865e-0c7b-4b88-95f9-738ebdbe5539 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:49.406503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8865e-0c7b-4b88-95f9-738ebdbe5539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:49.408165 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8865e-0c7b-4b88-95f9-738ebdbe5539 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:49.423319 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1070/2142] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:49.424510 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a7727-fdc5-4139-94ad-d72423226585 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:49.424766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8865e-0c7b-4b88-95f9-738ebdbe5539 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:49.426605 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-95c8865e-0c7b-4b88-95f9-738ebdbe5539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:49.437779 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a7727-fdc5-4139-94ad-d72423226585 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:49.440623 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8865e-0c7b-4b88-95f9-738ebdbe5539 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:49.444081 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f35890-0e4f-471c-b855-13a8c9c675b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:49.445201 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f35890-0e4f-471c-b855-13a8c9c675b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:49.445543 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f35890-0e4f-471c-b855-13a8c9c675b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:49.445799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f35890-0e4f-471c-b855-13a8c9c675b2 admin admin] PATH_INFO: `/v3/projects/5b771d4ccf534b53ae55a817db12a1c5/users/c18ed2c05dab4d1a8fd18ca0cb409dd2/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:49.449490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0f459-05b0-49ed-9492-34ebe5846709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:49.450361 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0f459-05b0-49ed-9492-34ebe5846709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:49.451252 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0f459-05b0-49ed-9492-34ebe5846709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:49.451548 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0f459-05b0-49ed-9492-34ebe5846709 admin admin] PATH_INFO: `/v3/projects/f9708567c672427da37c4dc9f5e02b42` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:49.452787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0f459-05b0-49ed-9492-34ebe5846709 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9708567c672427da37c4dc9f5e02b42)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:49.454882 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0f459-05b0-49ed-9492-34ebe5846709 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:49.462825 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:49.463674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f35890-0e4f-471c-b855-13a8c9c675b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b771d4ccf534b53ae55a817db12a1c5, user_id=c18ed2c05dab4d1a8fd18ca0cb409dd2, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:49.466669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f35890-0e4f-471c-b855-13a8c9c675b2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:49.480411 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1073/2143] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:49 2022] PUT /identity/v3/projects/5b771d4ccf534b53ae55a817db12a1c5/users/c18ed2c05dab4d1a8fd18ca0cb409dd2/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:49.491785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36be51-176b-48ca-8f13-5cbbb468e6a7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:49.492130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36be51-176b-48ca-8f13-5cbbb468e6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:49.492415 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36be51-176b-48ca-8f13-5cbbb468e6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:49.505978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e7b0f459-05b0-49ed-9492-34ebe5846709 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9708567c672427da37c4dc9f5e02b42', 'request_id': 'req-e7b0f459-05b0-49ed-9492-34ebe5846709'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:49.511981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e7b0f459-05b0-49ed-9492-34ebe5846709 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9708567c672427da37c4dc9f5e02b42', 'request_id': 'req-e7b0f459-05b0-49ed-9492-34ebe5846709'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:49.519035 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0a36be51-176b-48ca-8f13-5cbbb468e6a7 None None] MFA Rules not processed for user `c18ed2c05dab4d1a8fd18ca0cb409dd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:49.519812 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e7b0f459-05b0-49ed-9492-34ebe5846709 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9708567c672427da37c4dc9f5e02b42', 'request_id': 'req-e7b0f459-05b0-49ed-9492-34ebe5846709'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:49.520336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0a36be51-176b-48ca-8f13-5cbbb468e6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:49.531849 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e7b0f459-05b0-49ed-9492-34ebe5846709 admin admin] The token cache is being invalidate because project f9708567c672427da37c4dc9f5e02b42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:49.532172 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e7b0f459-05b0-49ed-9492-34ebe5846709 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:49.532829 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e7b0f459-05b0-49ed-9492-34ebe5846709 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:49.534383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1071/2144] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:49 2022] DELETE /identity/v3/projects/f9708567c672427da37c4dc9f5e02b42 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:49.539500 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f15ba-bc1a-491a-9314-38bd8f3be018 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:49.539903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f15ba-bc1a-491a-9314-38bd8f3be018 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:49.540226 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f15ba-bc1a-491a-9314-38bd8f3be018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:49.557639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-876f15ba-bc1a-491a-9314-38bd8f3be018 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:49.558839 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-876f15ba-bc1a-491a-9314-38bd8f3be018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:49.585825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1074/2145] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:49 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:09:49.590361 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36be51-176b-48ca-8f13-5cbbb468e6a7 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:49.591671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0a36be51-176b-48ca-8f13-5cbbb468e6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:49.597744 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:49.621072 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1072/2146] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:09:49.628589 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f39b8-732e-4cf3-ad89-a8146a3c3830 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:49.628886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f39b8-732e-4cf3-ad89-a8146a3c3830 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:49.629820 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f39b8-732e-4cf3-ad89-a8146a3c3830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:49.647296 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ab4f39b8-732e-4cf3-ad89-a8146a3c3830 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:49.648529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ab4f39b8-732e-4cf3-ad89-a8146a3c3830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:49.656346 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1073/2147] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:09:49.659532 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36be51-176b-48ca-8f13-5cbbb468e6a7 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:49.660609 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f39b8-732e-4cf3-ad89-a8146a3c3830 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:49.665056 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537c73b-e6fd-4fdf-868a-2d41d0ed891f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:49.665733 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537c73b-e6fd-4fdf-868a-2d41d0ed891f admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:49.666027 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537c73b-e6fd-4fdf-868a-2d41d0ed891f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:49.666314 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537c73b-e6fd-4fdf-868a-2d41d0ed891f admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:49.667020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9537c73b-e6fd-4fdf-868a-2d41d0ed891f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:49.669076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9537c73b-e6fd-4fdf-868a-2d41d0ed891f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:49.670230 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f39b8-732e-4cf3-ad89-a8146a3c3830 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:49.674822 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153226df-c101-4fda-b3f2-ea320e30a142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:49.675476 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153226df-c101-4fda-b3f2-ea320e30a142 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:49.675766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153226df-c101-4fda-b3f2-ea320e30a142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:49.676093 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153226df-c101-4fda-b3f2-ea320e30a142 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:49.676825 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153226df-c101-4fda-b3f2-ea320e30a142 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:49.677137 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153226df-c101-4fda-b3f2-ea320e30a142 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:49.679278 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153226df-c101-4fda-b3f2-ea320e30a142 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:49.691335 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1074/2148] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:09:49.696219 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153226df-c101-4fda-b3f2-ea320e30a142 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:49.699974 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1075/2149] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:49 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:09:49.704074 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537c73b-e6fd-4fdf-868a-2d41d0ed891f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:49.706182 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153226df-c101-4fda-b3f2-ea320e30a142 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:49.773421 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de184a1-f810-40f3-9e94-edb836e366e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:49.773985 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537c73b-e6fd-4fdf-868a-2d41d0ed891f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:49.774357 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de184a1-f810-40f3-9e94-edb836e366e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:49.774740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de184a1-f810-40f3-9e94-edb836e366e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:49.775119 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de184a1-f810-40f3-9e94-edb836e366e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:49.775912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de184a1-f810-40f3-9e94-edb836e366e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:49.777955 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de184a1-f810-40f3-9e94-edb836e366e3 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:49.778775 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd505c54-b7b6-4445-94ac-16f6fe4e71d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:49.779449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd505c54-b7b6-4445-94ac-16f6fe4e71d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:49.779744 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd505c54-b7b6-4445-94ac-16f6fe4e71d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:49.780067 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd505c54-b7b6-4445-94ac-16f6fe4e71d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:49.780958 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd505c54-b7b6-4445-94ac-16f6fe4e71d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:49.783524 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd505c54-b7b6-4445-94ac-16f6fe4e71d5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:49.800275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:49.811569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1076/2150] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:49 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:49.812050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1075/2151] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:49 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:09:49.812765 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd505c54-b7b6-4445-94ac-16f6fe4e71d5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:49.813821 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de184a1-f810-40f3-9e94-edb836e366e3 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:49.825777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd505c54-b7b6-4445-94ac-16f6fe4e71d5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:49.883420 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de184a1-f810-40f3-9e94-edb836e366e3 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:49.888274 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5ad19-2e6b-4d88-81ea-d6094120be14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:49.888948 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5ad19-2e6b-4d88-81ea-d6094120be14 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:49.889235 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5ad19-2e6b-4d88-81ea-d6094120be14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:49.889517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5ad19-2e6b-4d88-81ea-d6094120be14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:49.890372 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5ad19-2e6b-4d88-81ea-d6094120be14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:49.891844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5ad19-2e6b-4d88-81ea-d6094120be14 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:49.894688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a32bf-5d2f-43db-8fd4-9e2ecf6df6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:49.895468 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a32bf-5d2f-43db-8fd4-9e2ecf6df6a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:49.895827 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a32bf-5d2f-43db-8fd4-9e2ecf6df6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:49.896211 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a32bf-5d2f-43db-8fd4-9e2ecf6df6a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:49.897005 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a32bf-5d2f-43db-8fd4-9e2ecf6df6a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:49.899303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a32bf-5d2f-43db-8fd4-9e2ecf6df6a5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:49.903844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1076/2152] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:49.907790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5ad19-2e6b-4d88-81ea-d6094120be14 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:49.918451 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5ad19-2e6b-4d88-81ea-d6094120be14 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:49.923499 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2a5ce-5d43-49fc-aaaf-6f67b75403c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:49.924500 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2a5ce-5d43-49fc-aaaf-6f67b75403c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:49.925088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2a5ce-5d43-49fc-aaaf-6f67b75403c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:49.925539 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1077/2153] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:49 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:09:49.925678 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2a5ce-5d43-49fc-aaaf-6f67b75403c7 admin admin] PATH_INFO: `/v3/projects/96564e9f7f61444d87f42d58c7482071/users/d75d0a3c964845e1a0f236434fedd401/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:49.930724 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a32bf-5d2f-43db-8fd4-9e2ecf6df6a5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:49.946411 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2a5ce-5d43-49fc-aaaf-6f67b75403c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96564e9f7f61444d87f42d58c7482071, user_id=d75d0a3c964845e1a0f236434fedd401, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:49.949531 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2a5ce-5d43-49fc-aaaf-6f67b75403c7 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:49.954277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a32bf-5d2f-43db-8fd4-9e2ecf6df6a5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:49.960262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ce5d8-e76e-46d8-b97a-af0b220c3f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:49.961182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ce5d8-e76e-46d8-b97a-af0b220c3f3d admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:49.961562 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ce5d8-e76e-46d8-b97a-af0b220c3f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:49.961993 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ce5d8-e76e-46d8-b97a-af0b220c3f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:49.962744 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93ce5d8-e76e-46d8-b97a-af0b220c3f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:49.964440 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93ce5d8-e76e-46d8-b97a-af0b220c3f3d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:49.966882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:49.969412 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1077/2154] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:49 2022] PUT /identity/v3/projects/96564e9f7f61444d87f42d58c7482071/users/d75d0a3c964845e1a0f236434fedd401/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:09:49.974217 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2a5ce-5d43-49fc-aaaf-6f67b75403c7 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:49.982375 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1078/2155] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:09:49.985859 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ce5d8-e76e-46d8-b97a-af0b220c3f3d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:50.002376 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ce5d8-e76e-46d8-b97a-af0b220c3f3d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:50.053390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2a5ce-5d43-49fc-aaaf-6f67b75403c7 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:50.060502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d8b61-480a-4e54-b489-3c2ec9edf1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:50.061166 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d8b61-480a-4e54-b489-3c2ec9edf1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:50.061449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d8b61-480a-4e54-b489-3c2ec9edf1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:50.062034 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d8b61-480a-4e54-b489-3c2ec9edf1d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:50.063113 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d8b61-480a-4e54-b489-3c2ec9edf1d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:50.064756 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d8b61-480a-4e54-b489-3c2ec9edf1d8 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:50.072859 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc49b5-6c78-4fc8-9c8b-df287aa85a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:50.073554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc49b5-6c78-4fc8-9c8b-df287aa85a71 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:50.073841 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc49b5-6c78-4fc8-9c8b-df287aa85a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:50.074117 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc49b5-6c78-4fc8-9c8b-df287aa85a71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:50.074844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc49b5-6c78-4fc8-9c8b-df287aa85a71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:50.076402 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1078/2156] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:50.076520 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc49b5-6c78-4fc8-9c8b-df287aa85a71 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:50.077361 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d8b61-480a-4e54-b489-3c2ec9edf1d8 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:50.079013 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-249d8b61-480a-4e54-b489-3c2ec9edf1d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:50.087399 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1079/2157] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:09:50.090672 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc49b5-6c78-4fc8-9c8b-df287aa85a71 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:50.092032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d8b61-480a-4e54-b489-3c2ec9edf1d8 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:50.096575 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873bd8a0-ba72-4ed8-9403-8974dae44788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:50.097420 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873bd8a0-ba72-4ed8-9403-8974dae44788 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:50.097818 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873bd8a0-ba72-4ed8-9403-8974dae44788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:50.098099 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873bd8a0-ba72-4ed8-9403-8974dae44788 admin admin] PATH_INFO: `/v3/users/d41d7740b55e42ffa3e7a015c41bdbf9` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:50.099292 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873bd8a0-ba72-4ed8-9403-8974dae44788 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d41d7740b55e42ffa3e7a015c41bdbf9)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:50.100641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc49b5-6c78-4fc8-9c8b-df287aa85a71 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:50.101248 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873bd8a0-ba72-4ed8-9403-8974dae44788 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:50.105855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6390e-6102-41ba-918f-c231154eb5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:50.106722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6390e-6102-41ba-918f-c231154eb5b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:50.107000 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6390e-6102-41ba-918f-c231154eb5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:50.107305 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6390e-6102-41ba-918f-c231154eb5b2 admin admin] PATH_INFO: `/v3/projects/96564e9f7f61444d87f42d58c7482071/users/d75d0a3c964845e1a0f236434fedd401/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:50.109339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6390e-6102-41ba-918f-c231154eb5b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96564e9f7f61444d87f42d58c7482071, user_id=d75d0a3c964845e1a0f236434fedd401, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:50.112163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6390e-6102-41ba-918f-c231154eb5b2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:50.122816 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1080/2158] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:50 2022] PUT /identity/v3/projects/96564e9f7f61444d87f42d58c7482071/users/d75d0a3c964845e1a0f236434fedd401/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:50.123789 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6390e-6102-41ba-918f-c231154eb5b2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:50.155511 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-873bd8a0-ba72-4ed8-9403-8974dae44788 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd41d7740b55e42ffa3e7a015c41bdbf9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:50.163186 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-873bd8a0-ba72-4ed8-9403-8974dae44788 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd41d7740b55e42ffa3e7a015c41bdbf9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:50.170545 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-873bd8a0-ba72-4ed8-9403-8974dae44788 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd41d7740b55e42ffa3e7a015c41bdbf9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:50.171197 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-873bd8a0-ba72-4ed8-9403-8974dae44788 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd41d7740b55e42ffa3e7a015c41bdbf9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:50.172382 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:50.186004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-873bd8a0-ba72-4ed8-9403-8974dae44788 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd41d7740b55e42ffa3e7a015c41bdbf9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:50.192472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1079/2159] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:50 2022] DELETE /identity/v3/users/d41d7740b55e42ffa3e7a015c41bdbf9 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:50.194297 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e1f84-f4e7-4b09-a8b4-b2162ea76a0b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:50.194746 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e1f84-f4e7-4b09-a8b4-b2162ea76a0b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:50.195033 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e1f84-f4e7-4b09-a8b4-b2162ea76a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:50.199655 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6390e-6102-41ba-918f-c231154eb5b2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:50.221178 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-9b1e1f84-f4e7-4b09-a8b4-b2162ea76a0b None None] MFA Rules not processed for user `d75d0a3c964845e1a0f236434fedd401`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:50.222379 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9b1e1f84-f4e7-4b09-a8b4-b2162ea76a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:50.255562 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a2d01-2182-4faf-8116-6bd431cd5958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:50.257008 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a2d01-2182-4faf-8116-6bd431cd5958 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:50.257385 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a2d01-2182-4faf-8116-6bd431cd5958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:50.257702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a2d01-2182-4faf-8116-6bd431cd5958 admin admin] PATH_INFO: `/v3/projects/f4c2a7d2b75d496c973060aa5617e366/users/3ea6d66f608243368c0b8d34c907a493/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:50.272442 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a2d01-2182-4faf-8116-6bd431cd5958 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4c2a7d2b75d496c973060aa5617e366, user_id=3ea6d66f608243368c0b8d34c907a493, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:50.275141 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a2d01-2182-4faf-8116-6bd431cd5958 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:50.292055 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1080/2160] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:50 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:50.296665 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf942f32-a4f6-412e-85a4-18df5cb34014 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:50.296665 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf942f32-a4f6-412e-85a4-18df5cb34014 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:50.296665 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf942f32-a4f6-412e-85a4-18df5cb34014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:50.299581 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1081/2161] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:50 2022] PUT /identity/v3/projects/f4c2a7d2b75d496c973060aa5617e366/users/3ea6d66f608243368c0b8d34c907a493/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:50.310168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda2619-065d-45d5-8b50-9147fbef4472 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:50.310469 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda2619-065d-45d5-8b50-9147fbef4472 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:50.310753 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda2619-065d-45d5-8b50-9147fbef4472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:50.317978 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-bf942f32-a4f6-412e-85a4-18df5cb34014 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:50.319772 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bf942f32-a4f6-412e-85a4-18df5cb34014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:50.331447 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:50.343250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bcda2619-065d-45d5-8b50-9147fbef4472 None None] MFA Rules not processed for user `3ea6d66f608243368c0b8d34c907a493`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:50.344718 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bcda2619-065d-45d5-8b50-9147fbef4472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:50.393721 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1081/2162] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:50.394880 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf942f32-a4f6-412e-85a4-18df5cb34014 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:50.396397 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bf942f32-a4f6-412e-85a4-18df5cb34014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:50.421361 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1082/2163] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:09:50.422420 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda2619-065d-45d5-8b50-9147fbef4472 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:50.424358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bcda2619-065d-45d5-8b50-9147fbef4472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:50.465373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf942f32-a4f6-412e-85a4-18df5cb34014 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:50.469555 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be7448-13dd-4651-b79f-6bcf1ab23ef1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:50.470396 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be7448-13dd-4651-b79f-6bcf1ab23ef1 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:50.470993 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be7448-13dd-4651-b79f-6bcf1ab23ef1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:50.471114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be7448-13dd-4651-b79f-6bcf1ab23ef1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:50.488966 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda2619-065d-45d5-8b50-9147fbef4472 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:50.491878 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:50.494696 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73a21d-bfb2-4920-8331-923b54e99053 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:50.495438 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73a21d-bfb2-4920-8331-923b54e99053 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:50.495876 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73a21d-bfb2-4920-8331-923b54e99053 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:50.496454 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73a21d-bfb2-4920-8331-923b54e99053 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:50.511128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b73a21d-bfb2-4920-8331-923b54e99053 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:50.513614 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b73a21d-bfb2-4920-8331-923b54e99053 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:50.525786 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1083/2164] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:50.527293 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73a21d-bfb2-4920-8331-923b54e99053 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:50.529072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4b73a21d-bfb2-4920-8331-923b54e99053 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:50.541591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73a21d-bfb2-4920-8331-923b54e99053 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:50.544684 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be7448-13dd-4651-b79f-6bcf1ab23ef1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:50.545549 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e374c-8830-4d22-9566-7dd53b801308 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:50.546202 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e374c-8830-4d22-9566-7dd53b801308 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:50.546490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e374c-8830-4d22-9566-7dd53b801308 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:50.546763 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e374c-8830-4d22-9566-7dd53b801308 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:50.547086 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be7448-13dd-4651-b79f-6bcf1ab23ef1 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:50.558032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1082/2165] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:50 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:50.559843 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e374c-8830-4d22-9566-7dd53b801308 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:50.561320 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e374c-8830-4d22-9566-7dd53b801308 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:50.571172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1084/2166] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:50.656807 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be7448-13dd-4651-b79f-6bcf1ab23ef1 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:50.668493 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be7448-13dd-4651-b79f-6bcf1ab23ef1 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:50.674494 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5733d3b-ded8-41e7-8a53-22d1eab7df34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:50.675692 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5733d3b-ded8-41e7-8a53-22d1eab7df34 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:50.676327 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5733d3b-ded8-41e7-8a53-22d1eab7df34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:50.676619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5733d3b-ded8-41e7-8a53-22d1eab7df34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:50.679400 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b5733d3b-ded8-41e7-8a53-22d1eab7df34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:50.762525 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5733d3b-ded8-41e7-8a53-22d1eab7df34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:50.764043 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5733d3b-ded8-41e7-8a53-22d1eab7df34 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:50.775937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1083/2167] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:50 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:50.962381 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e374c-8830-4d22-9566-7dd53b801308 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:50.963847 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6f6e374c-8830-4d22-9566-7dd53b801308 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:51.001407 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e374c-8830-4d22-9566-7dd53b801308 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:51.007165 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392861b9-ff9f-4b00-a4e1-babd03f56215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:51.008175 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392861b9-ff9f-4b00-a4e1-babd03f56215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:51.008537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392861b9-ff9f-4b00-a4e1-babd03f56215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:51.008906 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392861b9-ff9f-4b00-a4e1-babd03f56215 admin admin] PATH_INFO: `/v3/projects/fe97437d94f249f1830c92d98832f5ff` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:51.010160 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392861b9-ff9f-4b00-a4e1-babd03f56215 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe97437d94f249f1830c92d98832f5ff)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:51.013945 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392861b9-ff9f-4b00-a4e1-babd03f56215 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:51.074132 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-392861b9-ff9f-4b00-a4e1-babd03f56215 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe97437d94f249f1830c92d98832f5ff', 'request_id': 'req-392861b9-ff9f-4b00-a4e1-babd03f56215'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:51.080620 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-392861b9-ff9f-4b00-a4e1-babd03f56215 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe97437d94f249f1830c92d98832f5ff', 'request_id': 'req-392861b9-ff9f-4b00-a4e1-babd03f56215'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:51.090533 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-392861b9-ff9f-4b00-a4e1-babd03f56215 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe97437d94f249f1830c92d98832f5ff', 'request_id': 'req-392861b9-ff9f-4b00-a4e1-babd03f56215'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:51.103931 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-392861b9-ff9f-4b00-a4e1-babd03f56215 admin admin] The token cache is being invalidate because project fe97437d94f249f1830c92d98832f5ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:51.104334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-392861b9-ff9f-4b00-a4e1-babd03f56215 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:51.105281 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-392861b9-ff9f-4b00-a4e1-babd03f56215 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:51.108039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1085/2168] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:50 2022] DELETE /identity/v3/projects/fe97437d94f249f1830c92d98832f5ff => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:51.111842 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5733d3b-ded8-41e7-8a53-22d1eab7df34 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:51.131852 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5733d3b-ded8-41e7-8a53-22d1eab7df34 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:51.202440 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ca22c-a0c3-4728-8980-dd8437648443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:51.203642 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ca22c-a0c3-4728-8980-dd8437648443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:51.203933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ca22c-a0c3-4728-8980-dd8437648443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:51.204232 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ca22c-a0c3-4728-8980-dd8437648443 admin admin] PATH_INFO: `/v3/users/aa7edbcedf594068ab006e90e0e154a3` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:51.205495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ca22c-a0c3-4728-8980-dd8437648443 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa7edbcedf594068ab006e90e0e154a3)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:51.207469 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ca22c-a0c3-4728-8980-dd8437648443 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:51.264325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d47ca22c-a0c3-4728-8980-dd8437648443 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa7edbcedf594068ab006e90e0e154a3'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:51.271998 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d47ca22c-a0c3-4728-8980-dd8437648443 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa7edbcedf594068ab006e90e0e154a3'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:51.274117 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:51.281780 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d47ca22c-a0c3-4728-8980-dd8437648443 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa7edbcedf594068ab006e90e0e154a3'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:51.282680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d47ca22c-a0c3-4728-8980-dd8437648443 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa7edbcedf594068ab006e90e0e154a3'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:51.295282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d47ca22c-a0c3-4728-8980-dd8437648443 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa7edbcedf594068ab006e90e0e154a3'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:51.301371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1084/2169] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:09:51 2022] DELETE /identity/v3/users/aa7edbcedf594068ab006e90e0e154a3 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:51.572855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392861b9-ff9f-4b00-a4e1-babd03f56215 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:51.574345 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-392861b9-ff9f-4b00-a4e1-babd03f56215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:51.587197 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392861b9-ff9f-4b00-a4e1-babd03f56215 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:51.596955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:51.646565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b0a91-d6cd-45bd-a692-2cd21ff93c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:51.647460 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b0a91-d6cd-45bd-a692-2cd21ff93c01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:51.647747 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b0a91-d6cd-45bd-a692-2cd21ff93c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:51.648057 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b0a91-d6cd-45bd-a692-2cd21ff93c01 admin admin] PATH_INFO: `/v3/projects/394863796f5749aaafa760342b3bc1c0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:51.649228 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b0a91-d6cd-45bd-a692-2cd21ff93c01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=394863796f5749aaafa760342b3bc1c0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:51.651135 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b0a91-d6cd-45bd-a692-2cd21ff93c01 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:51.696347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fc7b0a91-d6cd-45bd-a692-2cd21ff93c01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '394863796f5749aaafa760342b3bc1c0', 'request_id': 'req-fc7b0a91-d6cd-45bd-a692-2cd21ff93c01'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:51.702426 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fc7b0a91-d6cd-45bd-a692-2cd21ff93c01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '394863796f5749aaafa760342b3bc1c0', 'request_id': 'req-fc7b0a91-d6cd-45bd-a692-2cd21ff93c01'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:51.710187 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fc7b0a91-d6cd-45bd-a692-2cd21ff93c01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '394863796f5749aaafa760342b3bc1c0', 'request_id': 'req-fc7b0a91-d6cd-45bd-a692-2cd21ff93c01'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:51.725710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fc7b0a91-d6cd-45bd-a692-2cd21ff93c01 admin admin] The token cache is being invalidate because project 394863796f5749aaafa760342b3bc1c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:09:51.726008 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fc7b0a91-d6cd-45bd-a692-2cd21ff93c01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:51.726673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fc7b0a91-d6cd-45bd-a692-2cd21ff93c01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:09:51.728270 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1086/2170] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:09:51 2022] DELETE /identity/v3/projects/394863796f5749aaafa760342b3bc1c0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:51.734194 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c58389-ee44-4f46-981d-0515cbda731d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:51.734676 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c58389-ee44-4f46-981d-0515cbda731d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:51.734780 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c58389-ee44-4f46-981d-0515cbda731d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:51.753423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f7c58389-ee44-4f46-981d-0515cbda731d None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:51.754634 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f7c58389-ee44-4f46-981d-0515cbda731d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:51.813946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1085/2171] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:51.822859 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac0e8e-57da-45f8-8385-e1503d3159da None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:51.823162 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac0e8e-57da-45f8-8385-e1503d3159da None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:51.823480 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac0e8e-57da-45f8-8385-e1503d3159da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:51.842020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-acac0e8e-57da-45f8-8385-e1503d3159da None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:51.843234 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-acac0e8e-57da-45f8-8385-e1503d3159da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:51.851793 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1087/2172] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:51.855901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c58389-ee44-4f46-981d-0515cbda731d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:51.866264 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c58389-ee44-4f46-981d-0515cbda731d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:51.870842 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89fa64-f82f-448d-b750-85319e33182f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:51.871495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89fa64-f82f-448d-b750-85319e33182f admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:51.871786 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89fa64-f82f-448d-b750-85319e33182f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:51.872118 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89fa64-f82f-448d-b750-85319e33182f admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:51.872853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc89fa64-f82f-448d-b750-85319e33182f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:51.873165 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc89fa64-f82f-448d-b750-85319e33182f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:51.875086 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc89fa64-f82f-448d-b750-85319e33182f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:51.887142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1086/2173] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:51.891534 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac0e8e-57da-45f8-8385-e1503d3159da None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:51.901120 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac0e8e-57da-45f8-8385-e1503d3159da None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:51.906855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682a588-6b5a-4686-bdf8-5c1afebd3f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:51.907538 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682a588-6b5a-4686-bdf8-5c1afebd3f7c admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:51.907830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682a588-6b5a-4686-bdf8-5c1afebd3f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:51.908149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682a588-6b5a-4686-bdf8-5c1afebd3f7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:51.908863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4682a588-6b5a-4686-bdf8-5c1afebd3f7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:51.910776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4682a588-6b5a-4686-bdf8-5c1afebd3f7c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:51.937667 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1088/2174] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:09:51 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:51.951462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89fa64-f82f-448d-b750-85319e33182f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:51.962210 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89fa64-f82f-448d-b750-85319e33182f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:52.036687 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df92e72-3bca-42cc-adb1-c900ca449849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:52.037634 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df92e72-3bca-42cc-adb1-c900ca449849 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:52.037937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df92e72-3bca-42cc-adb1-c900ca449849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:52.044425 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df92e72-3bca-42cc-adb1-c900ca449849 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:52.045578 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df92e72-3bca-42cc-adb1-c900ca449849 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:52.048808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df92e72-3bca-42cc-adb1-c900ca449849 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:52.079260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1087/2175] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:09:51 2022] POST /identity/v3/users => generated 286 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:52.083474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682a588-6b5a-4686-bdf8-5c1afebd3f7c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:52.100476 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682a588-6b5a-4686-bdf8-5c1afebd3f7c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:52.105751 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a20b9-6084-419b-85c2-bfa75df46372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:52.106427 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a20b9-6084-419b-85c2-bfa75df46372 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:52.106814 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a20b9-6084-419b-85c2-bfa75df46372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:52.107116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a20b9-6084-419b-85c2-bfa75df46372 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:52.107746 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a20b9-6084-419b-85c2-bfa75df46372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:52.109467 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a20b9-6084-419b-85c2-bfa75df46372 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:52.137838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1089/2176] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:52.139869 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df92e72-3bca-42cc-adb1-c900ca449849 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:52.152596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df92e72-3bca-42cc-adb1-c900ca449849 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:52.159538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d969c-c54b-4f17-89b0-ba95220188a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:52.161592 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d969c-c54b-4f17-89b0-ba95220188a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:52.162217 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d969c-c54b-4f17-89b0-ba95220188a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:52.162398 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d969c-c54b-4f17-89b0-ba95220188a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:52.163147 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d969c-c54b-4f17-89b0-ba95220188a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:52.164896 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d969c-c54b-4f17-89b0-ba95220188a9 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:52.183276 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1088/2177] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:52.187550 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a20b9-6084-419b-85c2-bfa75df46372 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:52.202661 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a20b9-6084-419b-85c2-bfa75df46372 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:52.207753 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1580ad7-9406-4ba3-a961-4988058c7156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:52.209387 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1580ad7-9406-4ba3-a961-4988058c7156 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:52.209705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1580ad7-9406-4ba3-a961-4988058c7156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:52.210073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1580ad7-9406-4ba3-a961-4988058c7156 admin admin] PATH_INFO: `/v3/projects/2e0380aba6504b61af9e82a6a72d0211/users/936b6d72268244d09c079532a24a969c/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:52.231086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1580ad7-9406-4ba3-a961-4988058c7156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e0380aba6504b61af9e82a6a72d0211, user_id=936b6d72268244d09c079532a24a969c, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:52.233843 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1580ad7-9406-4ba3-a961-4988058c7156 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:52.254225 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1090/2178] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:09:52 2022] PUT /identity/v3/projects/2e0380aba6504b61af9e82a6a72d0211/users/936b6d72268244d09c079532a24a969c/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:52.259348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97c5e8-3ae7-4597-bd20-419f179a6341 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:52.259699 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97c5e8-3ae7-4597-bd20-419f179a6341 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:52.259926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97c5e8-3ae7-4597-bd20-419f179a6341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:52.293303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-da97c5e8-3ae7-4597-bd20-419f179a6341 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:52.294575 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-da97c5e8-3ae7-4597-bd20-419f179a6341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:52.339185 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:52.372576 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1089/2179] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:52.382390 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1580ad7-9406-4ba3-a961-4988058c7156 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:52.383870 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d1580ad7-9406-4ba3-a961-4988058c7156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:52.455706 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:52.481479 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1580ad7-9406-4ba3-a961-4988058c7156 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:52.485635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7e660-0b69-4c89-b14e-179efab0cd34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:52.486588 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7e660-0b69-4c89-b14e-179efab0cd34 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:52.486885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7e660-0b69-4c89-b14e-179efab0cd34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:52.487172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7e660-0b69-4c89-b14e-179efab0cd34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:52.498312 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7e660-0b69-4c89-b14e-179efab0cd34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:52.499970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7e660-0b69-4c89-b14e-179efab0cd34 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:52.511844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1091/2180] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:09:53.109844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d63a84-05c7-4be7-9d04-ad5a60df15b4 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:53.110301 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d63a84-05c7-4be7-9d04-ad5a60df15b4 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:53.110422 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d63a84-05c7-4be7-9d04-ad5a60df15b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:53.137022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-30d63a84-05c7-4be7-9d04-ad5a60df15b4 None None] MFA Rules not processed for user `936b6d72268244d09c079532a24a969c`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:53.138278 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-30d63a84-05c7-4be7-9d04-ad5a60df15b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:53.208533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1090/2181] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:09:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:09:53.220680 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7e660-0b69-4c89-b14e-179efab0cd34 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:53.222131 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d4c7e660-0b69-4c89-b14e-179efab0cd34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:53.280996 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7e660-0b69-4c89-b14e-179efab0cd34 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:53.284643 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ce5b8-9c6e-4276-9748-c365cc5dc66d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:53.285293 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ce5b8-9c6e-4276-9748-c365cc5dc66d service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:53.285583 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ce5b8-9c6e-4276-9748-c365cc5dc66d service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:53.285865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ce5b8-9c6e-4276-9748-c365cc5dc66d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:53.294737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ce5b8-9c6e-4276-9748-c365cc5dc66d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:53.296217 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ce5b8-9c6e-4276-9748-c365cc5dc66d service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:53.305371 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1092/2182] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:09:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:10:00.089784 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9766c-249f-4f4c-9d00-fc33a4d5f0ef None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:00.090286 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9766c-249f-4f4c-9d00-fc33a4d5f0ef None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:00.090415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9766c-249f-4f4c-9d00-fc33a4d5f0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:00.108796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-42b9766c-249f-4f4c-9d00-fc33a4d5f0ef None None] MFA Rules not processed for user `936b6d72268244d09c079532a24a969c`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:00.109981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-42b9766c-249f-4f4c-9d00-fc33a4d5f0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:00.117150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1091/2183] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:10:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:10:03.301139 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=34 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:04.419961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ce5b8-9c6e-4276-9748-c365cc5dc66d service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:04.422278 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-352ce5b8-9c6e-4276-9748-c365cc5dc66d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:04.438300 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ce5b8-9c6e-4276-9748-c365cc5dc66d service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:04.443741 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858e725-4a2e-441f-a1f0-822fe6bcbabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:04.444696 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858e725-4a2e-441f-a1f0-822fe6bcbabd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:04.444989 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858e725-4a2e-441f-a1f0-822fe6bcbabd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:04.445293 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858e725-4a2e-441f-a1f0-822fe6bcbabd admin admin] PATH_INFO: `/v3/users/3ea6d66f608243368c0b8d34c907a493` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:04.446580 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858e725-4a2e-441f-a1f0-822fe6bcbabd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ea6d66f608243368c0b8d34c907a493)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:04.448638 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858e725-4a2e-441f-a1f0-822fe6bcbabd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:04.510861 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a858e725-4a2e-441f-a1f0-822fe6bcbabd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ea6d66f608243368c0b8d34c907a493'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:04.519136 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a858e725-4a2e-441f-a1f0-822fe6bcbabd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ea6d66f608243368c0b8d34c907a493'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:04.527759 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a858e725-4a2e-441f-a1f0-822fe6bcbabd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ea6d66f608243368c0b8d34c907a493'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:04.528541 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a858e725-4a2e-441f-a1f0-822fe6bcbabd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ea6d66f608243368c0b8d34c907a493'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:04.544187 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a858e725-4a2e-441f-a1f0-822fe6bcbabd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ea6d66f608243368c0b8d34c907a493'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:04.550901 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1093/2184] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:10:04 2022] DELETE /identity/v3/users/3ea6d66f608243368c0b8d34c907a493 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:10:04.555878 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9a6f8-7526-4373-91e7-e6c6489e4877 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:04.556394 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9a6f8-7526-4373-91e7-e6c6489e4877 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:04.556492 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9a6f8-7526-4373-91e7-e6c6489e4877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:04.579442 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-06e9a6f8-7526-4373-91e7-e6c6489e4877 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:04.584329 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-06e9a6f8-7526-4373-91e7-e6c6489e4877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:04.671442 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1092/2185] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:10:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:10:04.682313 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858e725-4a2e-441f-a1f0-822fe6bcbabd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:04.683764 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a858e725-4a2e-441f-a1f0-822fe6bcbabd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:04.782144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858e725-4a2e-441f-a1f0-822fe6bcbabd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:04.790636 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b8e14-be2c-4e10-83b4-2f38bf3efbc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:04.792518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b8e14-be2c-4e10-83b4-2f38bf3efbc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:04.792816 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b8e14-be2c-4e10-83b4-2f38bf3efbc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:04.793107 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b8e14-be2c-4e10-83b4-2f38bf3efbc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:04.820704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b8e14-be2c-4e10-83b4-2f38bf3efbc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:04.822973 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b8e14-be2c-4e10-83b4-2f38bf3efbc2 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:04.843618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1094/2186] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:10:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:10:05.546663 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9a6f8-7526-4373-91e7-e6c6489e4877 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:05.548436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-06e9a6f8-7526-4373-91e7-e6c6489e4877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:05.572788 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9a6f8-7526-4373-91e7-e6c6489e4877 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:05.582276 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e004479-b56b-4d3c-bae0-bf0dcf793633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:05.584119 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e004479-b56b-4d3c-bae0-bf0dcf793633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:05.585119 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e004479-b56b-4d3c-bae0-bf0dcf793633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:05.585744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e004479-b56b-4d3c-bae0-bf0dcf793633 admin admin] PATH_INFO: `/v3/projects/f4c2a7d2b75d496c973060aa5617e366` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:05.587981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e004479-b56b-4d3c-bae0-bf0dcf793633 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4c2a7d2b75d496c973060aa5617e366)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:05.591182 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e004479-b56b-4d3c-bae0-bf0dcf793633 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:05.651391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0e004479-b56b-4d3c-bae0-bf0dcf793633 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4c2a7d2b75d496c973060aa5617e366', 'request_id': 'req-0e004479-b56b-4d3c-bae0-bf0dcf793633'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:05.657741 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0e004479-b56b-4d3c-bae0-bf0dcf793633 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4c2a7d2b75d496c973060aa5617e366', 'request_id': 'req-0e004479-b56b-4d3c-bae0-bf0dcf793633'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:05.659639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:05.667833 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0e004479-b56b-4d3c-bae0-bf0dcf793633 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4c2a7d2b75d496c973060aa5617e366', 'request_id': 'req-0e004479-b56b-4d3c-bae0-bf0dcf793633'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:05.682161 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0e004479-b56b-4d3c-bae0-bf0dcf793633 admin admin] The token cache is being invalidate because project f4c2a7d2b75d496c973060aa5617e366 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:10:05.682560 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0e004479-b56b-4d3c-bae0-bf0dcf793633 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:05.683299 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0e004479-b56b-4d3c-bae0-bf0dcf793633 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:05.685076 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1093/2187] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:10:05 2022] DELETE /identity/v3/projects/f4c2a7d2b75d496c973060aa5617e366 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:10:05.690420 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b227c6-942c-4730-9cba-a74ef57d97a0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:05.691050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b227c6-942c-4730-9cba-a74ef57d97a0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:05.691157 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b227c6-942c-4730-9cba-a74ef57d97a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:05.709938 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c1b227c6-942c-4730-9cba-a74ef57d97a0 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:05.711306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c1b227c6-942c-4730-9cba-a74ef57d97a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:05.778770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1095/2188] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:10:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:10:05.787051 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec999da-56d2-4a3a-8fb9-c69b5033d2ad None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:05.787285 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec999da-56d2-4a3a-8fb9-c69b5033d2ad None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:05.787573 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec999da-56d2-4a3a-8fb9-c69b5033d2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:05.806374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-0ec999da-56d2-4a3a-8fb9-c69b5033d2ad None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:05.807589 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0ec999da-56d2-4a3a-8fb9-c69b5033d2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:05.821726 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1094/2189] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:10:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:10:05.825848 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b227c6-942c-4730-9cba-a74ef57d97a0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:05.836100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b227c6-942c-4730-9cba-a74ef57d97a0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:05.842039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2c0e5-0308-4f6c-ad52-cda29688b09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:05.842718 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2c0e5-0308-4f6c-ad52-cda29688b09b admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:05.843016 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2c0e5-0308-4f6c-ad52-cda29688b09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:05.843304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2c0e5-0308-4f6c-ad52-cda29688b09b admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:05.844249 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2c0e5-0308-4f6c-ad52-cda29688b09b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:10:05.844565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2c0e5-0308-4f6c-ad52-cda29688b09b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:05.846377 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2c0e5-0308-4f6c-ad52-cda29688b09b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:05.860933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1096/2190] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:10:05.865471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec999da-56d2-4a3a-8fb9-c69b5033d2ad None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:05.875131 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec999da-56d2-4a3a-8fb9-c69b5033d2ad None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:05.880013 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f0465-61da-412e-a372-b3f3b551c3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:05.880664 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f0465-61da-412e-a372-b3f3b551c3db admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:05.880957 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f0465-61da-412e-a372-b3f3b551c3db admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:05.881546 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f0465-61da-412e-a372-b3f3b551c3db admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:05.882546 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f0465-61da-412e-a372-b3f3b551c3db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:05.884582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f0465-61da-412e-a372-b3f3b551c3db admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:05.920199 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1095/2191] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:10:05 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:10:05.924318 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2c0e5-0308-4f6c-ad52-cda29688b09b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:05.930200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:05.936054 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2c0e5-0308-4f6c-ad52-cda29688b09b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:06.011016 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455ffd0-2db6-4de6-a695-47f54e196c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:06.011697 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455ffd0-2db6-4de6-a695-47f54e196c1b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:06.012013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455ffd0-2db6-4de6-a695-47f54e196c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:06.012299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455ffd0-2db6-4de6-a695-47f54e196c1b admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:06.013018 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455ffd0-2db6-4de6-a695-47f54e196c1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:06.014963 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455ffd0-2db6-4de6-a695-47f54e196c1b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:06.036954 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1097/2192] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:10:05 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:10:06.041458 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f0465-61da-412e-a372-b3f3b551c3db admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:06.052472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f0465-61da-412e-a372-b3f3b551c3db admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:06.057190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218d0b2-084a-496d-a4cb-e80e5e1d3b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:06.057854 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218d0b2-084a-496d-a4cb-e80e5e1d3b11 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:06.058145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218d0b2-084a-496d-a4cb-e80e5e1d3b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:06.058452 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218d0b2-084a-496d-a4cb-e80e5e1d3b11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:06.059582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218d0b2-084a-496d-a4cb-e80e5e1d3b11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:06.060787 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218d0b2-084a-496d-a4cb-e80e5e1d3b11 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:06.072557 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1096/2193] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:10:06.076543 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455ffd0-2db6-4de6-a695-47f54e196c1b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:06.087504 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455ffd0-2db6-4de6-a695-47f54e196c1b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:06.092319 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46767af7-ee4c-4d98-a34f-9ded6ba9e22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:06.092985 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46767af7-ee4c-4d98-a34f-9ded6ba9e22d admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:06.093276 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46767af7-ee4c-4d98-a34f-9ded6ba9e22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:06.093557 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46767af7-ee4c-4d98-a34f-9ded6ba9e22d admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:06.094296 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46767af7-ee4c-4d98-a34f-9ded6ba9e22d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:06.095895 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46767af7-ee4c-4d98-a34f-9ded6ba9e22d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:06.106983 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1098/2194] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:10:06.111854 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218d0b2-084a-496d-a4cb-e80e5e1d3b11 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:06.125315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218d0b2-084a-496d-a4cb-e80e5e1d3b11 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:06.130235 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da7a7c-8333-46b9-b9de-37b8699372c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:06.131156 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da7a7c-8333-46b9-b9de-37b8699372c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:06.131446 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da7a7c-8333-46b9-b9de-37b8699372c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:06.131728 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da7a7c-8333-46b9-b9de-37b8699372c0 admin admin] PATH_INFO: `/v3/projects/76ac92496e0c475194469fddc97f05f4/users/be30fe38545548dab0e1a1007d37ea98/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:06.149487 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da7a7c-8333-46b9-b9de-37b8699372c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76ac92496e0c475194469fddc97f05f4, user_id=be30fe38545548dab0e1a1007d37ea98, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:06.152263 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da7a7c-8333-46b9-b9de-37b8699372c0 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:06.166768 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1097/2195] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:10:06 2022] PUT /identity/v3/projects/76ac92496e0c475194469fddc97f05f4/users/be30fe38545548dab0e1a1007d37ea98/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:10:06.176778 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a3ad2-d083-4194-8942-3e8b8e852473 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:06.177114 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a3ad2-d083-4194-8942-3e8b8e852473 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:06.177422 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a3ad2-d083-4194-8942-3e8b8e852473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:06.205302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a57a3ad2-d083-4194-8942-3e8b8e852473 None None] MFA Rules not processed for user `be30fe38545548dab0e1a1007d37ea98`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:06.208260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a57a3ad2-d083-4194-8942-3e8b8e852473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:06.294302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1099/2196] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:10:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:10:06.304446 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da7a7c-8333-46b9-b9de-37b8699372c0 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:06.306276 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-50da7a7c-8333-46b9-b9de-37b8699372c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:06.374714 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da7a7c-8333-46b9-b9de-37b8699372c0 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:06.378648 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4054d-eb86-41e5-bb7f-4be7b16777bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:06.379311 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4054d-eb86-41e5-bb7f-4be7b16777bd service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:06.379631 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4054d-eb86-41e5-bb7f-4be7b16777bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:06.381085 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4054d-eb86-41e5-bb7f-4be7b16777bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:06.393021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e4054d-eb86-41e5-bb7f-4be7b16777bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:06.394664 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e4054d-eb86-41e5-bb7f-4be7b16777bd service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:06.404730 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1098/2197] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:10:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:10:16.286552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=65,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:16.400739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=57 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:46.328740 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9965291-64ce-4c68-97ac-9b07c4dc574d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:46.329257 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9965291-64ce-4c68-97ac-9b07c4dc574d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:46.329361 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9965291-64ce-4c68-97ac-9b07c4dc574d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:46.348019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c9965291-64ce-4c68-97ac-9b07c4dc574d None None] MFA Rules not processed for user `936b6d72268244d09c079532a24a969c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:46.349230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c9965291-64ce-4c68-97ac-9b07c4dc574d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:46.409228 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1100/2198] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:10:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:10:48.511723 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4054d-eb86-41e5-bb7f-4be7b16777bd service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:48.577500 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4054d-eb86-41e5-bb7f-4be7b16777bd service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:48.582653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5bf8d-335a-40fb-b36b-a49646e998e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:48.583544 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5bf8d-335a-40fb-b36b-a49646e998e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:48.583837 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5bf8d-335a-40fb-b36b-a49646e998e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:48.584147 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5bf8d-335a-40fb-b36b-a49646e998e6 admin admin] PATH_INFO: `/v3/users/be30fe38545548dab0e1a1007d37ea98` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:48.585368 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa5bf8d-335a-40fb-b36b-a49646e998e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be30fe38545548dab0e1a1007d37ea98)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:48.587312 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa5bf8d-335a-40fb-b36b-a49646e998e6 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:48.635645 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-aaa5bf8d-335a-40fb-b36b-a49646e998e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be30fe38545548dab0e1a1007d37ea98'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:48.644943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-aaa5bf8d-335a-40fb-b36b-a49646e998e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be30fe38545548dab0e1a1007d37ea98'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:48.652313 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-aaa5bf8d-335a-40fb-b36b-a49646e998e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be30fe38545548dab0e1a1007d37ea98'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:48.652977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-aaa5bf8d-335a-40fb-b36b-a49646e998e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be30fe38545548dab0e1a1007d37ea98'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:48.665470 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-aaa5bf8d-335a-40fb-b36b-a49646e998e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be30fe38545548dab0e1a1007d37ea98'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:48.671832 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1099/2199] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:10:48 2022] DELETE /identity/v3/users/be30fe38545548dab0e1a1007d37ea98 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:10:48.676416 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d843e2-f4a3-4cd5-891e-ee8d1f0f4598 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:48.676962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d843e2-f4a3-4cd5-891e-ee8d1f0f4598 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:48.676962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d843e2-f4a3-4cd5-891e-ee8d1f0f4598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:48.697453 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-87d843e2-f4a3-4cd5-891e-ee8d1f0f4598 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:48.698669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-87d843e2-f4a3-4cd5-891e-ee8d1f0f4598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:48.764924 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1101/2200] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:10:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:10:48.779224 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5bf8d-335a-40fb-b36b-a49646e998e6 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:48.781440 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-aaa5bf8d-335a-40fb-b36b-a49646e998e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:48.847231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5bf8d-335a-40fb-b36b-a49646e998e6 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:48.851552 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39919c-4d49-4015-ac2f-d5f80395c8ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:48.852261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39919c-4d49-4015-ac2f-d5f80395c8ac service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:48.852555 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39919c-4d49-4015-ac2f-d5f80395c8ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:48.853903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39919c-4d49-4015-ac2f-d5f80395c8ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:48.863994 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39919c-4d49-4015-ac2f-d5f80395c8ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:48.865505 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39919c-4d49-4015-ac2f-d5f80395c8ac service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:48.875661 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1100/2201] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:10:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:10:49.320473 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d843e2-f4a3-4cd5-891e-ee8d1f0f4598 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:49.321962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-87d843e2-f4a3-4cd5-891e-ee8d1f0f4598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:49.334289 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d843e2-f4a3-4cd5-891e-ee8d1f0f4598 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:49.339251 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255e8ead-6608-41d8-94a9-47d8fb0a044b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:49.340751 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e8ead-6608-41d8-94a9-47d8fb0a044b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:49.341200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e8ead-6608-41d8-94a9-47d8fb0a044b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:49.341641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e8ead-6608-41d8-94a9-47d8fb0a044b admin admin] PATH_INFO: `/v3/projects/76ac92496e0c475194469fddc97f05f4` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:49.344053 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e8ead-6608-41d8-94a9-47d8fb0a044b admin admin] RBAC: Authorizing `identity:delete_project(project_id=76ac92496e0c475194469fddc97f05f4)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:49.346515 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255e8ead-6608-41d8-94a9-47d8fb0a044b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:49.406160 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-255e8ead-6608-41d8-94a9-47d8fb0a044b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76ac92496e0c475194469fddc97f05f4', 'request_id': 'req-255e8ead-6608-41d8-94a9-47d8fb0a044b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:49.414658 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-255e8ead-6608-41d8-94a9-47d8fb0a044b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76ac92496e0c475194469fddc97f05f4', 'request_id': 'req-255e8ead-6608-41d8-94a9-47d8fb0a044b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:49.423045 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-255e8ead-6608-41d8-94a9-47d8fb0a044b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76ac92496e0c475194469fddc97f05f4', 'request_id': 'req-255e8ead-6608-41d8-94a9-47d8fb0a044b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:49.434039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-255e8ead-6608-41d8-94a9-47d8fb0a044b admin admin] The token cache is being invalidate because project 76ac92496e0c475194469fddc97f05f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:10:49.434336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-255e8ead-6608-41d8-94a9-47d8fb0a044b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:49.434999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-255e8ead-6608-41d8-94a9-47d8fb0a044b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:49.436603 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1102/2202] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:10:49 2022] DELETE /identity/v3/projects/76ac92496e0c475194469fddc97f05f4 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:10:49.442362 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf28cf4-a840-4729-b5f1-90960f39be8f None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:49.442666 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf28cf4-a840-4729-b5f1-90960f39be8f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:49.442945 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf28cf4-a840-4729-b5f1-90960f39be8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:49.465782 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-cbf28cf4-a840-4729-b5f1-90960f39be8f None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:49.467446 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-cbf28cf4-a840-4729-b5f1-90960f39be8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:49.486304 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:49.532376 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1101/2203] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:10:49.541817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b239e12-008a-4fa7-8925-f1fbb47f7f5e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:49.542196 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b239e12-008a-4fa7-8925-f1fbb47f7f5e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:49.542545 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b239e12-008a-4fa7-8925-f1fbb47f7f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:49.564382 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3b239e12-008a-4fa7-8925-f1fbb47f7f5e None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:49.565697 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3b239e12-008a-4fa7-8925-f1fbb47f7f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:49.573911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1103/2204] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:10:49.578067 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf28cf4-a840-4729-b5f1-90960f39be8f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:49.589406 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf28cf4-a840-4729-b5f1-90960f39be8f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:49.595114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da332565-a981-40d9-a765-747aeb5a52bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:49.595782 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da332565-a981-40d9-a765-747aeb5a52bc admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:49.596118 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da332565-a981-40d9-a765-747aeb5a52bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:49.596401 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da332565-a981-40d9-a765-747aeb5a52bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:49.597169 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da332565-a981-40d9-a765-747aeb5a52bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:10:49.597985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da332565-a981-40d9-a765-747aeb5a52bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:49.599594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da332565-a981-40d9-a765-747aeb5a52bc admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:49.615750 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1102/2205] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:10:49.616965 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b239e12-008a-4fa7-8925-f1fbb47f7f5e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:49.626402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b239e12-008a-4fa7-8925-f1fbb47f7f5e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:49.631392 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59b2d0-2ce2-44bd-a5b1-f4afd6c025e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:49.631695 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59b2d0-2ce2-44bd-a5b1-f4afd6c025e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:49.632013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59b2d0-2ce2-44bd-a5b1-f4afd6c025e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:49.632294 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59b2d0-2ce2-44bd-a5b1-f4afd6c025e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:49.633163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a59b2d0-2ce2-44bd-a5b1-f4afd6c025e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:49.635070 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a59b2d0-2ce2-44bd-a5b1-f4afd6c025e4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:49.654470 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:49.662687 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1104/2206] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:10:49 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:10:49.667030 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da332565-a981-40d9-a765-747aeb5a52bc admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:49.677838 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da332565-a981-40d9-a765-747aeb5a52bc admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:49.738620 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424976cd-3a9d-4fd6-a414-174c8024f067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:49.739232 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424976cd-3a9d-4fd6-a414-174c8024f067 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:49.739526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424976cd-3a9d-4fd6-a414-174c8024f067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:49.739814 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424976cd-3a9d-4fd6-a414-174c8024f067 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:49.740546 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424976cd-3a9d-4fd6-a414-174c8024f067 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:49.742488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424976cd-3a9d-4fd6-a414-174c8024f067 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:49.764222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1103/2207] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:10:49 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:10:49.768545 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59b2d0-2ce2-44bd-a5b1-f4afd6c025e4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:49.779966 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59b2d0-2ce2-44bd-a5b1-f4afd6c025e4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:49.785039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44643985-9be5-4233-a79b-19e37e1776f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:49.785720 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44643985-9be5-4233-a79b-19e37e1776f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:49.786013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44643985-9be5-4233-a79b-19e37e1776f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:49.786298 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44643985-9be5-4233-a79b-19e37e1776f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:49.787054 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44643985-9be5-4233-a79b-19e37e1776f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:49.788697 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44643985-9be5-4233-a79b-19e37e1776f9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:49.800922 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1105/2208] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:10:49.804814 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424976cd-3a9d-4fd6-a414-174c8024f067 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:49.815747 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424976cd-3a9d-4fd6-a414-174c8024f067 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:49.821089 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b04d5a-2195-44df-a82d-1d0c3133328d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:49.821770 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b04d5a-2195-44df-a82d-1d0c3133328d admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:49.822062 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b04d5a-2195-44df-a82d-1d0c3133328d admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:49.822348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b04d5a-2195-44df-a82d-1d0c3133328d admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:49.823248 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b04d5a-2195-44df-a82d-1d0c3133328d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:49.824931 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b04d5a-2195-44df-a82d-1d0c3133328d admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:49.838446 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1104/2209] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:10:49.839749 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44643985-9be5-4233-a79b-19e37e1776f9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:49.852267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44643985-9be5-4233-a79b-19e37e1776f9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:49.857182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a985050-0bd8-4292-a7b5-ec3dbd26c838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:49.858233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a985050-0bd8-4292-a7b5-ec3dbd26c838 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:49.858522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a985050-0bd8-4292-a7b5-ec3dbd26c838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:49.858804 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a985050-0bd8-4292-a7b5-ec3dbd26c838 admin admin] PATH_INFO: `/v3/projects/2709afb81c074156b75f2407c0240cbf/users/6efd0742141b4124b02df2a345507ed9/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:49.877272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a985050-0bd8-4292-a7b5-ec3dbd26c838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2709afb81c074156b75f2407c0240cbf, user_id=6efd0742141b4124b02df2a345507ed9, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:49.881315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a985050-0bd8-4292-a7b5-ec3dbd26c838 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:49.894421 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1106/2210] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:10:49 2022] PUT /identity/v3/projects/2709afb81c074156b75f2407c0240cbf/users/6efd0742141b4124b02df2a345507ed9/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:10:49.904935 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4ee6a-48b8-4e2f-a8cc-2e7e83c9459a None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:49.905312 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4ee6a-48b8-4e2f-a8cc-2e7e83c9459a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:49.905676 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4ee6a-48b8-4e2f-a8cc-2e7e83c9459a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:49.934609 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-7fc4ee6a-48b8-4e2f-a8cc-2e7e83c9459a None None] MFA Rules not processed for user `6efd0742141b4124b02df2a345507ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:49.935923 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7fc4ee6a-48b8-4e2f-a8cc-2e7e83c9459a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:50.012688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1105/2211] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:10:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:10:50.024242 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a985050-0bd8-4292-a7b5-ec3dbd26c838 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:50.026075 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5a985050-0bd8-4292-a7b5-ec3dbd26c838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:50.092958 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a985050-0bd8-4292-a7b5-ec3dbd26c838 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:50.100367 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f27a0-26c3-491b-9df4-afad8a6bfe4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:50.101070 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f27a0-26c3-491b-9df4-afad8a6bfe4d service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:50.101365 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f27a0-26c3-491b-9df4-afad8a6bfe4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:50.101650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f27a0-26c3-491b-9df4-afad8a6bfe4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:50.112624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f27a0-26c3-491b-9df4-afad8a6bfe4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:50.114825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f27a0-26c3-491b-9df4-afad8a6bfe4d service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:50.125317 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1107/2212] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:10:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:10:57.347844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4ee6a-48b8-4e2f-a8cc-2e7e83c9459a None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:57.367202 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:57.411745 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4ee6a-48b8-4e2f-a8cc-2e7e83c9459a None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:57.416313 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73304b81-12b8-444a-8249-2967b696ef3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:57.417186 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73304b81-12b8-444a-8249-2967b696ef3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:57.417478 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73304b81-12b8-444a-8249-2967b696ef3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:57.417761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73304b81-12b8-444a-8249-2967b696ef3f admin admin] PATH_INFO: `/v3/users/6efd0742141b4124b02df2a345507ed9` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:57.418943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73304b81-12b8-444a-8249-2967b696ef3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6efd0742141b4124b02df2a345507ed9)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:57.420919 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73304b81-12b8-444a-8249-2967b696ef3f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:57.466067 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-73304b81-12b8-444a-8249-2967b696ef3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6efd0742141b4124b02df2a345507ed9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:57.473612 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-73304b81-12b8-444a-8249-2967b696ef3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6efd0742141b4124b02df2a345507ed9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:57.481075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-73304b81-12b8-444a-8249-2967b696ef3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6efd0742141b4124b02df2a345507ed9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:57.481718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-73304b81-12b8-444a-8249-2967b696ef3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6efd0742141b4124b02df2a345507ed9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:57.493563 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-73304b81-12b8-444a-8249-2967b696ef3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6efd0742141b4124b02df2a345507ed9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:57.499504 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1106/2213] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:10:57 2022] DELETE /identity/v3/users/6efd0742141b4124b02df2a345507ed9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:10:57.504508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8d1fb-8cab-4227-a208-55a1b5064cc3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:57.504925 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8d1fb-8cab-4227-a208-55a1b5064cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:57.505199 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8d1fb-8cab-4227-a208-55a1b5064cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:57.522497 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6ea8d1fb-8cab-4227-a208-55a1b5064cc3 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:57.523690 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6ea8d1fb-8cab-4227-a208-55a1b5064cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:57.586032 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1108/2214] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:10:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:10:57.595743 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73304b81-12b8-444a-8249-2967b696ef3f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:57.597327 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-73304b81-12b8-444a-8249-2967b696ef3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:57.664612 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73304b81-12b8-444a-8249-2967b696ef3f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:57.668488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d52a08-7605-4f06-8447-d119a5de5b22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:57.669237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d52a08-7605-4f06-8447-d119a5de5b22 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:57.669586 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d52a08-7605-4f06-8447-d119a5de5b22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:57.669931 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d52a08-7605-4f06-8447-d119a5de5b22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:57.679158 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d52a08-7605-4f06-8447-d119a5de5b22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:57.680613 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d52a08-7605-4f06-8447-d119a5de5b22 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:57.691878 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1107/2215] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:10:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:10:58.073752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8d1fb-8cab-4227-a208-55a1b5064cc3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:58.075768 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6ea8d1fb-8cab-4227-a208-55a1b5064cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:58.097902 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8d1fb-8cab-4227-a208-55a1b5064cc3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:58.107584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60037b4b-f65c-496b-8253-6df7e5ab2949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:58.108520 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60037b4b-f65c-496b-8253-6df7e5ab2949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:58.108809 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60037b4b-f65c-496b-8253-6df7e5ab2949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:58.109091 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60037b4b-f65c-496b-8253-6df7e5ab2949 admin admin] PATH_INFO: `/v3/projects/2709afb81c074156b75f2407c0240cbf` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:58.110296 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60037b4b-f65c-496b-8253-6df7e5ab2949 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2709afb81c074156b75f2407c0240cbf)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:58.112248 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60037b4b-f65c-496b-8253-6df7e5ab2949 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:58.161337 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-60037b4b-f65c-496b-8253-6df7e5ab2949 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2709afb81c074156b75f2407c0240cbf', 'request_id': 'req-60037b4b-f65c-496b-8253-6df7e5ab2949'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:58.167473 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-60037b4b-f65c-496b-8253-6df7e5ab2949 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2709afb81c074156b75f2407c0240cbf', 'request_id': 'req-60037b4b-f65c-496b-8253-6df7e5ab2949'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:58.175357 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-60037b4b-f65c-496b-8253-6df7e5ab2949 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2709afb81c074156b75f2407c0240cbf', 'request_id': 'req-60037b4b-f65c-496b-8253-6df7e5ab2949'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:58.177583 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:58.188634 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-60037b4b-f65c-496b-8253-6df7e5ab2949 admin admin] The token cache is being invalidate because project 2709afb81c074156b75f2407c0240cbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:10:58.188928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-60037b4b-f65c-496b-8253-6df7e5ab2949 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:58.189560 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-60037b4b-f65c-496b-8253-6df7e5ab2949 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:10:58.191696 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1109/2216] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:10:58 2022] DELETE /identity/v3/projects/2709afb81c074156b75f2407c0240cbf => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:10:58.198423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ae50b-b211-42ba-b403-e006b947e238 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:58.198725 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ae50b-b211-42ba-b403-e006b947e238 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:58.199014 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ae50b-b211-42ba-b403-e006b947e238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:58.216344 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-507ae50b-b211-42ba-b403-e006b947e238 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:58.217544 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-507ae50b-b211-42ba-b403-e006b947e238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:58.252305 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:58.280316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1108/2217] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:10:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:10:58.288974 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b3855-5fb9-4c66-94d0-acd6398051a0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:58.289312 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b3855-5fb9-4c66-94d0-acd6398051a0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:58.289601 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b3855-5fb9-4c66-94d0-acd6398051a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:58.308125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a22b3855-5fb9-4c66-94d0-acd6398051a0 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:58.309591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a22b3855-5fb9-4c66-94d0-acd6398051a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:58.318317 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1110/2218] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:10:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:10:58.322356 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ae50b-b211-42ba-b403-e006b947e238 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:58.332941 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ae50b-b211-42ba-b403-e006b947e238 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:58.337500 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ddb1be-4afd-426d-b10d-555024e6ee64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:58.338149 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddb1be-4afd-426d-b10d-555024e6ee64 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:58.338433 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddb1be-4afd-426d-b10d-555024e6ee64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:58.338740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ddb1be-4afd-426d-b10d-555024e6ee64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:58.339467 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ddb1be-4afd-426d-b10d-555024e6ee64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:10:58.339799 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ddb1be-4afd-426d-b10d-555024e6ee64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:58.341400 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ddb1be-4afd-426d-b10d-555024e6ee64 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:58.352971 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1109/2219] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:10:58.357042 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b3855-5fb9-4c66-94d0-acd6398051a0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:58.366109 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b3855-5fb9-4c66-94d0-acd6398051a0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:58.370719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77dcda-c15d-4535-abd0-c10beb3e9fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:58.371394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77dcda-c15d-4535-abd0-c10beb3e9fee admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:58.371682 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77dcda-c15d-4535-abd0-c10beb3e9fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:58.371979 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77dcda-c15d-4535-abd0-c10beb3e9fee admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:58.372677 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c77dcda-c15d-4535-abd0-c10beb3e9fee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:58.374559 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c77dcda-c15d-4535-abd0-c10beb3e9fee admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:58.400757 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1111/2220] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:10:58 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:10:58.404779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ddb1be-4afd-426d-b10d-555024e6ee64 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:58.414844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ddb1be-4afd-426d-b10d-555024e6ee64 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:58.471207 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f02224e-58d8-4611-9f82-7c2b9aa59ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:58.472005 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02224e-58d8-4611-9f82-7c2b9aa59ae1 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:58.472411 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02224e-58d8-4611-9f82-7c2b9aa59ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:58.472693 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02224e-58d8-4611-9f82-7c2b9aa59ae1 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:58.473490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f02224e-58d8-4611-9f82-7c2b9aa59ae1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:58.475754 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f02224e-58d8-4611-9f82-7c2b9aa59ae1 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:58.495496 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1110/2221] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:10:58 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:10:58.499529 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77dcda-c15d-4535-abd0-c10beb3e9fee admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:58.510358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77dcda-c15d-4535-abd0-c10beb3e9fee admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:58.514949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87bf46b-c6cb-4eb1-bd16-f82970a5545a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:58.515599 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bf46b-c6cb-4eb1-bd16-f82970a5545a admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:58.515884 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bf46b-c6cb-4eb1-bd16-f82970a5545a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:58.516206 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bf46b-c6cb-4eb1-bd16-f82970a5545a admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:58.516928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87bf46b-c6cb-4eb1-bd16-f82970a5545a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:58.518505 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87bf46b-c6cb-4eb1-bd16-f82970a5545a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:58.529434 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1112/2222] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:10:58.533372 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f02224e-58d8-4611-9f82-7c2b9aa59ae1 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:58.544143 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f02224e-58d8-4611-9f82-7c2b9aa59ae1 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:58.548728 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639553a6-4541-4d37-85c0-7efe8dd604f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:58.549397 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639553a6-4541-4d37-85c0-7efe8dd604f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:58.549685 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639553a6-4541-4d37-85c0-7efe8dd604f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:58.550036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639553a6-4541-4d37-85c0-7efe8dd604f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:58.550761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639553a6-4541-4d37-85c0-7efe8dd604f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:58.552391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639553a6-4541-4d37-85c0-7efe8dd604f2 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:58.562138 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1111/2223] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:10:58.566903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87bf46b-c6cb-4eb1-bd16-f82970a5545a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:58.576605 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87bf46b-c6cb-4eb1-bd16-f82970a5545a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:58.581315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfabb382-0959-405e-8181-09a6759576b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:58.582149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfabb382-0959-405e-8181-09a6759576b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:58.582431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfabb382-0959-405e-8181-09a6759576b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:58.582731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfabb382-0959-405e-8181-09a6759576b0 admin admin] PATH_INFO: `/v3/projects/f618e7dff5d7411eb4d4cade50d68607/users/2aa4925080114a33b80e2ca588fbeb27/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:58.597690 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfabb382-0959-405e-8181-09a6759576b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f618e7dff5d7411eb4d4cade50d68607, user_id=2aa4925080114a33b80e2ca588fbeb27, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:58.600411 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfabb382-0959-405e-8181-09a6759576b0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:58.614704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1113/2224] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:10:58 2022] PUT /identity/v3/projects/f618e7dff5d7411eb4d4cade50d68607/users/2aa4925080114a33b80e2ca588fbeb27/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:10:58.620016 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab34e6d-7478-47fc-93dd-55a78043a899 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:58.620384 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab34e6d-7478-47fc-93dd-55a78043a899 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:58.620699 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab34e6d-7478-47fc-93dd-55a78043a899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:58.638719 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-1ab34e6d-7478-47fc-93dd-55a78043a899 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:58.640028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1ab34e6d-7478-47fc-93dd-55a78043a899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:58.699432 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1112/2225] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:10:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:10:58.709109 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfabb382-0959-405e-8181-09a6759576b0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:58.710930 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cfabb382-0959-405e-8181-09a6759576b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:58.771575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfabb382-0959-405e-8181-09a6759576b0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:58.775833 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a286ac7-e3f1-458b-b699-6737ff9aa450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:58.776512 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a286ac7-e3f1-458b-b699-6737ff9aa450 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:58.776807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a286ac7-e3f1-458b-b699-6737ff9aa450 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:58.777148 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a286ac7-e3f1-458b-b699-6737ff9aa450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:58.787023 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a286ac7-e3f1-458b-b699-6737ff9aa450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:58.788504 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a286ac7-e3f1-458b-b699-6737ff9aa450 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:58.798035 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1114/2226] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:10:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:00.918871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab34e6d-7478-47fc-93dd-55a78043a899 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:00.921245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1ab34e6d-7478-47fc-93dd-55a78043a899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:00.962060 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab34e6d-7478-47fc-93dd-55a78043a899 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:00.967015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e658053-254d-4056-bf95-1d050a6beba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:00.968365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e658053-254d-4056-bf95-1d050a6beba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:00.968729 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e658053-254d-4056-bf95-1d050a6beba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:00.969013 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e658053-254d-4056-bf95-1d050a6beba1 admin admin] PATH_INFO: `/v3/users/c18ed2c05dab4d1a8fd18ca0cb409dd2` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:00.970306 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e658053-254d-4056-bf95-1d050a6beba1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c18ed2c05dab4d1a8fd18ca0cb409dd2)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:00.972372 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e658053-254d-4056-bf95-1d050a6beba1 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:01.022531 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:01.030633 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4e658053-254d-4056-bf95-1d050a6beba1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c18ed2c05dab4d1a8fd18ca0cb409dd2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:01.039965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4e658053-254d-4056-bf95-1d050a6beba1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c18ed2c05dab4d1a8fd18ca0cb409dd2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:01.047563 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4e658053-254d-4056-bf95-1d050a6beba1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c18ed2c05dab4d1a8fd18ca0cb409dd2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:01.048186 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4e658053-254d-4056-bf95-1d050a6beba1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c18ed2c05dab4d1a8fd18ca0cb409dd2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:01.061694 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4e658053-254d-4056-bf95-1d050a6beba1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c18ed2c05dab4d1a8fd18ca0cb409dd2'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:01.068765 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1113/2227] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:11:00 2022] DELETE /identity/v3/users/c18ed2c05dab4d1a8fd18ca0cb409dd2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:01.073400 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330daca8-f87a-426b-8aee-2a07a24daa60 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:01.073887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330daca8-f87a-426b-8aee-2a07a24daa60 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:01.074009 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330daca8-f87a-426b-8aee-2a07a24daa60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:01.091179 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-330daca8-f87a-426b-8aee-2a07a24daa60 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:01.092548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-330daca8-f87a-426b-8aee-2a07a24daa60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:01.156564 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1115/2228] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:01.166448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e658053-254d-4056-bf95-1d050a6beba1 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:01.171519 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4e658053-254d-4056-bf95-1d050a6beba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:01.233855 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e658053-254d-4056-bf95-1d050a6beba1 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:01.238277 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a42aa9-bc0f-41a7-95b9-e72cdf021ed9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:01.238940 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a42aa9-bc0f-41a7-95b9-e72cdf021ed9 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:01.239211 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a42aa9-bc0f-41a7-95b9-e72cdf021ed9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:01.239488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a42aa9-bc0f-41a7-95b9-e72cdf021ed9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:01.248682 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a42aa9-bc0f-41a7-95b9-e72cdf021ed9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:01.250136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a42aa9-bc0f-41a7-95b9-e72cdf021ed9 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:01.260564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1114/2229] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:02.505572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330daca8-f87a-426b-8aee-2a07a24daa60 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:02.507858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-330daca8-f87a-426b-8aee-2a07a24daa60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:02.530445 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330daca8-f87a-426b-8aee-2a07a24daa60 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:02.538140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55d9f6-a6d9-4107-b57f-98b0cd4f9c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:02.539241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55d9f6-a6d9-4107-b57f-98b0cd4f9c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:02.539681 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55d9f6-a6d9-4107-b57f-98b0cd4f9c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:02.541143 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55d9f6-a6d9-4107-b57f-98b0cd4f9c38 admin admin] PATH_INFO: `/v3/projects/5b771d4ccf534b53ae55a817db12a1c5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:02.542801 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc55d9f6-a6d9-4107-b57f-98b0cd4f9c38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b771d4ccf534b53ae55a817db12a1c5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:02.544989 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc55d9f6-a6d9-4107-b57f-98b0cd4f9c38 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:02.585212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:02.614136 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fc55d9f6-a6d9-4107-b57f-98b0cd4f9c38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b771d4ccf534b53ae55a817db12a1c5', 'request_id': 'req-fc55d9f6-a6d9-4107-b57f-98b0cd4f9c38'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:02.621937 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fc55d9f6-a6d9-4107-b57f-98b0cd4f9c38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b771d4ccf534b53ae55a817db12a1c5', 'request_id': 'req-fc55d9f6-a6d9-4107-b57f-98b0cd4f9c38'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:02.649521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fc55d9f6-a6d9-4107-b57f-98b0cd4f9c38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b771d4ccf534b53ae55a817db12a1c5', 'request_id': 'req-fc55d9f6-a6d9-4107-b57f-98b0cd4f9c38'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:02.665347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fc55d9f6-a6d9-4107-b57f-98b0cd4f9c38 admin admin] The token cache is being invalidate because project 5b771d4ccf534b53ae55a817db12a1c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:11:02.666376 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fc55d9f6-a6d9-4107-b57f-98b0cd4f9c38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:02.667205 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fc55d9f6-a6d9-4107-b57f-98b0cd4f9c38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:02.670099 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1116/2230] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:11:02 2022] DELETE /identity/v3/projects/5b771d4ccf534b53ae55a817db12a1c5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:02.673894 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a42aa9-bc0f-41a7-95b9-e72cdf021ed9 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:02.691391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a42aa9-bc0f-41a7-95b9-e72cdf021ed9 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:02.797419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5331d0-f1b1-4c14-ae37-851b3b16b272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:02.798236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5331d0-f1b1-4c14-ae37-851b3b16b272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:02.798528 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5331d0-f1b1-4c14-ae37-851b3b16b272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:02.798809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5331d0-f1b1-4c14-ae37-851b3b16b272 admin admin] PATH_INFO: `/v3/users/d75d0a3c964845e1a0f236434fedd401` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:02.800209 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5331d0-f1b1-4c14-ae37-851b3b16b272 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d75d0a3c964845e1a0f236434fedd401)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:02.802205 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5331d0-f1b1-4c14-ae37-851b3b16b272 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:02.867108 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7d5331d0-f1b1-4c14-ae37-851b3b16b272 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd75d0a3c964845e1a0f236434fedd401'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:02.881179 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7d5331d0-f1b1-4c14-ae37-851b3b16b272 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd75d0a3c964845e1a0f236434fedd401'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:02.891420 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7d5331d0-f1b1-4c14-ae37-851b3b16b272 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd75d0a3c964845e1a0f236434fedd401'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:02.892250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7d5331d0-f1b1-4c14-ae37-851b3b16b272 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd75d0a3c964845e1a0f236434fedd401'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:02.894946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:02.913139 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-7d5331d0-f1b1-4c14-ae37-851b3b16b272 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd75d0a3c964845e1a0f236434fedd401'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:02.920222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1115/2231] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:11:02 2022] DELETE /identity/v3/users/d75d0a3c964845e1a0f236434fedd401 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:03.232840 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55d9f6-a6d9-4107-b57f-98b0cd4f9c38 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:03.236364 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fc55d9f6-a6d9-4107-b57f-98b0cd4f9c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:03.269650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55d9f6-a6d9-4107-b57f-98b0cd4f9c38 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:03.369550 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61bd03-1d60-4956-bb10-cfc460c1e258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:03.370542 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61bd03-1d60-4956-bb10-cfc460c1e258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:03.370835 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61bd03-1d60-4956-bb10-cfc460c1e258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:03.371433 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61bd03-1d60-4956-bb10-cfc460c1e258 admin admin] PATH_INFO: `/v3/projects/96564e9f7f61444d87f42d58c7482071` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:03.372960 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61bd03-1d60-4956-bb10-cfc460c1e258 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96564e9f7f61444d87f42d58c7482071)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:03.375206 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61bd03-1d60-4956-bb10-cfc460c1e258 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:03.477994 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1c61bd03-1d60-4956-bb10-cfc460c1e258 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96564e9f7f61444d87f42d58c7482071', 'request_id': 'req-1c61bd03-1d60-4956-bb10-cfc460c1e258'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:03.500265 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1c61bd03-1d60-4956-bb10-cfc460c1e258 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96564e9f7f61444d87f42d58c7482071', 'request_id': 'req-1c61bd03-1d60-4956-bb10-cfc460c1e258'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:03.516267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1c61bd03-1d60-4956-bb10-cfc460c1e258 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96564e9f7f61444d87f42d58c7482071', 'request_id': 'req-1c61bd03-1d60-4956-bb10-cfc460c1e258'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:03.538111 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1c61bd03-1d60-4956-bb10-cfc460c1e258 admin admin] The token cache is being invalidate because project 96564e9f7f61444d87f42d58c7482071 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:11:03.538414 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1c61bd03-1d60-4956-bb10-cfc460c1e258 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:03.539100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1c61bd03-1d60-4956-bb10-cfc460c1e258 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:03.541145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1117/2232] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:11:03 2022] DELETE /identity/v3/projects/96564e9f7f61444d87f42d58c7482071 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:03.546201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9507d-5adb-4242-bd63-b00c8f83b9a3 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:03.546703 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9507d-5adb-4242-bd63-b00c8f83b9a3 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:03.547165 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9507d-5adb-4242-bd63-b00c8f83b9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:03.590300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f8d9507d-5adb-4242-bd63-b00c8f83b9a3 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:03.592008 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f8d9507d-5adb-4242-bd63-b00c8f83b9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:03.704593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1116/2233] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:03.713006 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb60e00-299a-4bdd-8034-3e81857a84f8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:03.713306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb60e00-299a-4bdd-8034-3e81857a84f8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:03.713596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb60e00-299a-4bdd-8034-3e81857a84f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:03.772277 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10603a3-1c92-4588-9d72-f728ecb551e6 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:03.772576 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10603a3-1c92-4588-9d72-f728ecb551e6 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:03.772881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10603a3-1c92-4588-9d72-f728ecb551e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:03.828863 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d10603a3-1c92-4588-9d72-f728ecb551e6 None None] MFA Rules not processed for user `2aa4925080114a33b80e2ca588fbeb27`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:03.828863 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d10603a3-1c92-4588-9d72-f728ecb551e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:03.829099 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-acb60e00-299a-4bdd-8034-3e81857a84f8 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:03.829649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-acb60e00-299a-4bdd-8034-3e81857a84f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:03.838637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1118/2234] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:03.842553 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb60e00-299a-4bdd-8034-3e81857a84f8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:03.853351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb60e00-299a-4bdd-8034-3e81857a84f8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:03.859040 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d2c5e-d6b8-4a21-80da-c388e0c1ee85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:03.859596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d2c5e-d6b8-4a21-80da-c388e0c1ee85 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:03.859965 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d2c5e-d6b8-4a21-80da-c388e0c1ee85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:03.860202 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d2c5e-d6b8-4a21-80da-c388e0c1ee85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:03.860978 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65d2c5e-d6b8-4a21-80da-c388e0c1ee85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:11:03.861249 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65d2c5e-d6b8-4a21-80da-c388e0c1ee85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:03.863901 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65d2c5e-d6b8-4a21-80da-c388e0c1ee85 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:03.881740 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1119/2235] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:11:03.882739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d2c5e-d6b8-4a21-80da-c388e0c1ee85 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:03.896217 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d2c5e-d6b8-4a21-80da-c388e0c1ee85 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:03.900898 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d8dee-140c-4d3a-ac66-9ee7bd10a247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:03.901565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d8dee-140c-4d3a-ac66-9ee7bd10a247 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:03.901852 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d8dee-140c-4d3a-ac66-9ee7bd10a247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:03.902142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d8dee-140c-4d3a-ac66-9ee7bd10a247 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:03.902839 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d8dee-140c-4d3a-ac66-9ee7bd10a247 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:03.904841 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63d8dee-140c-4d3a-ac66-9ee7bd10a247 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:03.915092 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1117/2236] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:11:03.920294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10603a3-1c92-4588-9d72-f728ecb551e6 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:03.921975 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d10603a3-1c92-4588-9d72-f728ecb551e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:03.938389 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1120/2237] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:11:03 2022] POST /identity/v3/projects => generated 355 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:11:03.941018 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d8dee-140c-4d3a-ac66-9ee7bd10a247 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:03.947304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:03.950385 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10603a3-1c92-4588-9d72-f728ecb551e6 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:04.025709 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee21cf-9730-4477-b3c0-8152088c8199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:04.026386 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee21cf-9730-4477-b3c0-8152088c8199 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:04.026671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee21cf-9730-4477-b3c0-8152088c8199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:04.026949 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee21cf-9730-4477-b3c0-8152088c8199 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:04.027059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d8dee-140c-4d3a-ac66-9ee7bd10a247 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:04.027681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbee21cf-9730-4477-b3c0-8152088c8199 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:04.029643 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbee21cf-9730-4477-b3c0-8152088c8199 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:04.032047 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db017023-4188-4950-99d9-78d5d71729ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:04.032723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db017023-4188-4950-99d9-78d5d71729ac admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:04.033022 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db017023-4188-4950-99d9-78d5d71729ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:04.033304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db017023-4188-4950-99d9-78d5d71729ac admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:04.034576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db017023-4188-4950-99d9-78d5d71729ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:04.036583 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db017023-4188-4950-99d9-78d5d71729ac admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:04.045329 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:04.062688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1121/2238] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:11:03 2022] POST /identity/v3/users => generated 278 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:11:04.064260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1118/2239] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:11:03 2022] POST /identity/v3/projects => generated 371 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:11:04.065232 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee21cf-9730-4477-b3c0-8152088c8199 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:04.065980 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db017023-4188-4950-99d9-78d5d71729ac admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:04.077693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db017023-4188-4950-99d9-78d5d71729ac admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:04.141427 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4152d-821b-4fd4-9750-86acfc98ce40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:04.142365 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4152d-821b-4fd4-9750-86acfc98ce40 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:04.142688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4152d-821b-4fd4-9750-86acfc98ce40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:04.143351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4152d-821b-4fd4-9750-86acfc98ce40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:04.144166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f4152d-821b-4fd4-9750-86acfc98ce40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:04.146685 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f4152d-821b-4fd4-9750-86acfc98ce40 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:04.151434 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee21cf-9730-4477-b3c0-8152088c8199 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:04.157984 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15585b-4795-47b1-92cb-05d18dffb1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:04.158730 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15585b-4795-47b1-92cb-05d18dffb1ac admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:04.159043 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15585b-4795-47b1-92cb-05d18dffb1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:04.159338 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15585b-4795-47b1-92cb-05d18dffb1ac admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:04.160246 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1122/2240] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:11:04.163030 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b15585b-4795-47b1-92cb-05d18dffb1ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:04.163874 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4152d-821b-4fd4-9750-86acfc98ce40 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:04.165286 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b15585b-4795-47b1-92cb-05d18dffb1ac admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:04.178594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4152d-821b-4fd4-9750-86acfc98ce40 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:04.184552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01474735-7814-4c9c-938b-4bb7a34474d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:04.185688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01474735-7814-4c9c-938b-4bb7a34474d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:04.186204 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01474735-7814-4c9c-938b-4bb7a34474d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:04.186591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01474735-7814-4c9c-938b-4bb7a34474d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:04.188463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01474735-7814-4c9c-938b-4bb7a34474d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:04.190101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01474735-7814-4c9c-938b-4bb7a34474d1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:04.198252 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1119/2241] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:11:04 2022] POST /identity/v3/users => generated 292 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:11:04.203359 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15585b-4795-47b1-92cb-05d18dffb1ac admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:04.209556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1123/2242] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:11:04.216110 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01474735-7814-4c9c-938b-4bb7a34474d1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:04.229311 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15585b-4795-47b1-92cb-05d18dffb1ac admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:04.239337 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01474735-7814-4c9c-938b-4bb7a34474d1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:04.245464 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fbd649-04a0-4d13-9e01-5b81b790c9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:04.246222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fbd649-04a0-4d13-9e01-5b81b790c9e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:04.246531 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fbd649-04a0-4d13-9e01-5b81b790c9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:04.246811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fbd649-04a0-4d13-9e01-5b81b790c9e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:04.247589 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fbd649-04a0-4d13-9e01-5b81b790c9e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:04.256831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38640db-20d1-40d1-9db2-f9ebb04a6f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:04.257057 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fbd649-04a0-4d13-9e01-5b81b790c9e4 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:04.257760 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38640db-20d1-40d1-9db2-f9ebb04a6f77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:04.258065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38640db-20d1-40d1-9db2-f9ebb04a6f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:04.258345 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38640db-20d1-40d1-9db2-f9ebb04a6f77 admin admin] PATH_INFO: `/v3/projects/cfab1efff31f49b79d52a9b7cbb11b73/users/50650edbfbeb421592c87ff3b21ed132/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:04.267953 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1120/2243] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:11:04.272212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fbd649-04a0-4d13-9e01-5b81b790c9e4 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:04.274193 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38640db-20d1-40d1-9db2-f9ebb04a6f77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfab1efff31f49b79d52a9b7cbb11b73, user_id=50650edbfbeb421592c87ff3b21ed132, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:04.276966 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38640db-20d1-40d1-9db2-f9ebb04a6f77 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:04.291160 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fbd649-04a0-4d13-9e01-5b81b790c9e4 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:04.296888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1124/2244] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:11:04 2022] PUT /identity/v3/projects/cfab1efff31f49b79d52a9b7cbb11b73/users/50650edbfbeb421592c87ff3b21ed132/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:11:04.308117 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93912fc7-4b00-4d6f-adca-0c94e3cf15ad None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:04.308489 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93912fc7-4b00-4d6f-adca-0c94e3cf15ad None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:04.308841 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93912fc7-4b00-4d6f-adca-0c94e3cf15ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:04.339648 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-93912fc7-4b00-4d6f-adca-0c94e3cf15ad None None] MFA Rules not processed for user `50650edbfbeb421592c87ff3b21ed132`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:04.340964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-93912fc7-4b00-4d6f-adca-0c94e3cf15ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:04.357003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0647f-0db2-4f30-9e8b-51f78c8caa15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:04.357827 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0647f-0db2-4f30-9e8b-51f78c8caa15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:04.358121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0647f-0db2-4f30-9e8b-51f78c8caa15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:04.358408 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0647f-0db2-4f30-9e8b-51f78c8caa15 admin admin] PATH_INFO: `/v3/projects/d8b0bbd989be47ea916196e8f8b4d8f1/users/b7d5e46b2dcb4246a8267eb3654b9e74/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:04.381670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0647f-0db2-4f30-9e8b-51f78c8caa15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8b0bbd989be47ea916196e8f8b4d8f1, user_id=b7d5e46b2dcb4246a8267eb3654b9e74, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:04.384600 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0647f-0db2-4f30-9e8b-51f78c8caa15 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:04.392018 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:04.400076 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1121/2245] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:11:04 2022] PUT /identity/v3/projects/d8b0bbd989be47ea916196e8f8b4d8f1/users/b7d5e46b2dcb4246a8267eb3654b9e74/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:11:04.404284 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0647f-0db2-4f30-9e8b-51f78c8caa15 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:04.423441 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1125/2246] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:04 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:11:04.429461 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93912fc7-4b00-4d6f-adca-0c94e3cf15ad None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:04.445190 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93912fc7-4b00-4d6f-adca-0c94e3cf15ad None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:04.467744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:04.472097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0647f-0db2-4f30-9e8b-51f78c8caa15 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:04.478269 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067c20b-cd47-4026-8321-5443a2fa0108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:04.479176 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067c20b-cd47-4026-8321-5443a2fa0108 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:04.479454 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067c20b-cd47-4026-8321-5443a2fa0108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:04.479768 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067c20b-cd47-4026-8321-5443a2fa0108 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:04.480688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067c20b-cd47-4026-8321-5443a2fa0108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:04.482316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067c20b-cd47-4026-8321-5443a2fa0108 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:04.494686 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1122/2247] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:11:04.498933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067c20b-cd47-4026-8321-5443a2fa0108 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:04.510494 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067c20b-cd47-4026-8321-5443a2fa0108 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:04.515417 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b896d2e-7f57-4ab9-801c-30c9805cb546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:04.515566 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4746f9-dd8b-4536-921d-83bdcfbc7653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:04.516168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4746f9-dd8b-4536-921d-83bdcfbc7653 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:04.516360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b896d2e-7f57-4ab9-801c-30c9805cb546 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:04.516479 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4746f9-dd8b-4536-921d-83bdcfbc7653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:04.516911 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b896d2e-7f57-4ab9-801c-30c9805cb546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:04.516990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4746f9-dd8b-4536-921d-83bdcfbc7653 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:04.517072 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b896d2e-7f57-4ab9-801c-30c9805cb546 admin admin] PATH_INFO: `/v3/projects/d8b0bbd989be47ea916196e8f8b4d8f1/users/b7d5e46b2dcb4246a8267eb3654b9e74/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:04.517402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4746f9-dd8b-4536-921d-83bdcfbc7653 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:04.519031 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b896d2e-7f57-4ab9-801c-30c9805cb546 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8b0bbd989be47ea916196e8f8b4d8f1, user_id=b7d5e46b2dcb4246a8267eb3654b9e74, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:04.519815 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4746f9-dd8b-4536-921d-83bdcfbc7653 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:04.521886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b896d2e-7f57-4ab9-801c-30c9805cb546 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:04.533136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1123/2248] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:11:04 2022] PUT /identity/v3/projects/d8b0bbd989be47ea916196e8f8b4d8f1/users/b7d5e46b2dcb4246a8267eb3654b9e74/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:11:04.550003 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1126/2249] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:11:04 2022] POST /identity/v3/projects => generated 355 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:11:04.554018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b896d2e-7f57-4ab9-801c-30c9805cb546 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:04.634870 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b896d2e-7f57-4ab9-801c-30c9805cb546 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:04.709118 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33a1bf-2945-4b7f-9540-4e1d9620b781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:04.709779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33a1bf-2945-4b7f-9540-4e1d9620b781 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:04.710076 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33a1bf-2945-4b7f-9540-4e1d9620b781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:04.710313 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33a1bf-2945-4b7f-9540-4e1d9620b781 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:04.711280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb33a1bf-2945-4b7f-9540-4e1d9620b781 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:04.713690 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb33a1bf-2945-4b7f-9540-4e1d9620b781 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:04.755480 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1124/2250] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:11:04 2022] POST /identity/v3/users => generated 284 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:11:04.759684 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4746f9-dd8b-4536-921d-83bdcfbc7653 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:04.774043 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4746f9-dd8b-4536-921d-83bdcfbc7653 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:04.778840 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c1f53-d998-44f7-a7d7-70f7b523ca5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:04.779548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c1f53-d998-44f7-a7d7-70f7b523ca5d admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:04.779840 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c1f53-d998-44f7-a7d7-70f7b523ca5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:04.780155 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c1f53-d998-44f7-a7d7-70f7b523ca5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:04.780933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c1f53-d998-44f7-a7d7-70f7b523ca5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:04.782530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c1f53-d998-44f7-a7d7-70f7b523ca5d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:04.817425 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1127/2251] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:11:04.821818 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33a1bf-2945-4b7f-9540-4e1d9620b781 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:04.842153 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33a1bf-2945-4b7f-9540-4e1d9620b781 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:04.858408 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6defbe6e-ff53-495b-bbc1-6b48d528596f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:04.859329 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defbe6e-ff53-495b-bbc1-6b48d528596f admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:04.859615 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defbe6e-ff53-495b-bbc1-6b48d528596f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:04.859893 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defbe6e-ff53-495b-bbc1-6b48d528596f admin admin] PATH_INFO: `/v3/projects/933725adaa1741e98ab4d4f50c739616/users/7ca1ce6eb8d34837b7508325909e5b48/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:04.895817 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6defbe6e-ff53-495b-bbc1-6b48d528596f admin admin] RBAC: Authorizing `identity:create_grant(project_id=933725adaa1741e98ab4d4f50c739616, user_id=7ca1ce6eb8d34837b7508325909e5b48, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:04.904511 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6defbe6e-ff53-495b-bbc1-6b48d528596f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:04.922460 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1125/2252] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:11:04 2022] PUT /identity/v3/projects/933725adaa1741e98ab4d4f50c739616/users/7ca1ce6eb8d34837b7508325909e5b48/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:04.926671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c1f53-d998-44f7-a7d7-70f7b523ca5d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:04.939880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c1f53-d998-44f7-a7d7-70f7b523ca5d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:05.004703 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1323fa80-822c-431d-9702-d272201d7d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:05.005436 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323fa80-822c-431d-9702-d272201d7d0d admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:05.005794 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323fa80-822c-431d-9702-d272201d7d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:05.006143 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1323fa80-822c-431d-9702-d272201d7d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:05.006966 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1323fa80-822c-431d-9702-d272201d7d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:05.008933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1323fa80-822c-431d-9702-d272201d7d0d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:05.020902 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1128/2253] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:11:05.025501 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6defbe6e-ff53-495b-bbc1-6b48d528596f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:05.043633 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6defbe6e-ff53-495b-bbc1-6b48d528596f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:05.049687 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec759058-cf43-4af4-9783-bf85b5f77835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:05.051196 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec759058-cf43-4af4-9783-bf85b5f77835 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:05.051490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec759058-cf43-4af4-9783-bf85b5f77835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:05.051774 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec759058-cf43-4af4-9783-bf85b5f77835 admin admin] PATH_INFO: `/v3/projects/933725adaa1741e98ab4d4f50c739616/users/7ca1ce6eb8d34837b7508325909e5b48/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:05.054498 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec759058-cf43-4af4-9783-bf85b5f77835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=933725adaa1741e98ab4d4f50c739616, user_id=7ca1ce6eb8d34837b7508325909e5b48, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:05.056993 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec759058-cf43-4af4-9783-bf85b5f77835 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:05.071905 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1126/2254] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:11:05 2022] PUT /identity/v3/projects/933725adaa1741e98ab4d4f50c739616/users/7ca1ce6eb8d34837b7508325909e5b48/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:05.082604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a0f7c-4403-491a-a4ac-41a3e07eb506 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:05.082907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a0f7c-4403-491a-a4ac-41a3e07eb506 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:05.083180 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a0f7c-4403-491a-a4ac-41a3e07eb506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:05.118251 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-153a0f7c-4403-491a-a4ac-41a3e07eb506 None None] MFA Rules not processed for user `7ca1ce6eb8d34837b7508325909e5b48`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:05.119414 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-153a0f7c-4403-491a-a4ac-41a3e07eb506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:05.180530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:05.234947 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1129/2255] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:05 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:05.247521 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec759058-cf43-4af4-9783-bf85b5f77835 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:05.249398 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ec759058-cf43-4af4-9783-bf85b5f77835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:05.312211 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:05.342197 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec759058-cf43-4af4-9783-bf85b5f77835 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:05.347687 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69cd26-142d-4dc6-82dd-4e4f6d91f7b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:05.348335 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69cd26-142d-4dc6-82dd-4e4f6d91f7b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:05.349269 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69cd26-142d-4dc6-82dd-4e4f6d91f7b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:05.349671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69cd26-142d-4dc6-82dd-4e4f6d91f7b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:05.364335 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69cd26-142d-4dc6-82dd-4e4f6d91f7b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:05.366221 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69cd26-142d-4dc6-82dd-4e4f6d91f7b4 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:05.378893 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1127/2256] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:05 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:05.512963 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a0f7c-4403-491a-a4ac-41a3e07eb506 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:05.525202 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a0f7c-4403-491a-a4ac-41a3e07eb506 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:05.535043 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1131b4b-972c-47fa-8a0e-f67754f0b196 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:05.536194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1131b4b-972c-47fa-8a0e-f67754f0b196 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:05.536605 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1131b4b-972c-47fa-8a0e-f67754f0b196 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:05.536967 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1131b4b-972c-47fa-8a0e-f67754f0b196 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:05.659938 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1131b4b-972c-47fa-8a0e-f67754f0b196 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:05.661458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1131b4b-972c-47fa-8a0e-f67754f0b196 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:05.679575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1130/2257] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:05 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:10.672227 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f71f2b-e73f-4aa6-a2e3-6407e65b004c None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:10.672836 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f71f2b-e73f-4aa6-a2e3-6407e65b004c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:10.672836 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f71f2b-e73f-4aa6-a2e3-6407e65b004c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:10.709007 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-80f71f2b-e73f-4aa6-a2e3-6407e65b004c None None] MFA Rules not processed for user `b7d5e46b2dcb4246a8267eb3654b9e74`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:10.710892 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-80f71f2b-e73f-4aa6-a2e3-6407e65b004c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:10.795985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1128/2258] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:10 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:11:10.810701 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1131b4b-972c-47fa-8a0e-f67754f0b196 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:10.812975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c1131b4b-972c-47fa-8a0e-f67754f0b196 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:10.911221 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1131b4b-972c-47fa-8a0e-f67754f0b196 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:10.919796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172b6df-c530-4cd8-80bf-1013ed8ccbce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:10.920944 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172b6df-c530-4cd8-80bf-1013ed8ccbce service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:10.921238 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172b6df-c530-4cd8-80bf-1013ed8ccbce service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:10.921594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172b6df-c530-4cd8-80bf-1013ed8ccbce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:10.992661 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8172b6df-c530-4cd8-80bf-1013ed8ccbce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:10.994339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8172b6df-c530-4cd8-80bf-1013ed8ccbce service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:11.006646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1131/2259] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:12.187479 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7d040-9521-46bd-927d-3d249db1b72b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:12.188433 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7d040-9521-46bd-927d-3d249db1b72b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:12.188909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7d040-9521-46bd-927d-3d249db1b72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:12.225033 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-5df7d040-9521-46bd-927d-3d249db1b72b None None] MFA Rules not processed for user `2aa4925080114a33b80e2ca588fbeb27`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:12.226290 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5df7d040-9521-46bd-927d-3d249db1b72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:12.241009 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1129/2260] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:15.429698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bd807-8938-456c-a95d-256fbc4dd638 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:15.430194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bd807-8938-456c-a95d-256fbc4dd638 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:15.430302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bd807-8938-456c-a95d-256fbc4dd638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:15.448190 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-106bd807-8938-456c-a95d-256fbc4dd638 None None] MFA Rules not processed for user `3720c2225b51464abee34d05a69b9db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:15.449414 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-106bd807-8938-456c-a95d-256fbc4dd638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:15.458452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:15.508158 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1132/2261] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:22.203098 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=45 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:25.500228 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=19 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:36.100192 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26ef5d-ed64-4424-bf66-fb75fb2f124d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:36.100643 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26ef5d-ed64-4424-bf66-fb75fb2f124d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:36.100750 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26ef5d-ed64-4424-bf66-fb75fb2f124d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:36.118419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ff26ef5d-ed64-4424-bf66-fb75fb2f124d None None] MFA Rules not processed for user `94d824bffc2c4af299c537056bc21734`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:36.119614 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ff26ef5d-ed64-4424-bf66-fb75fb2f124d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:36.173245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1130/2262] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Jul 18 14:11:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:36.214490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106bd807-8938-456c-a95d-256fbc4dd638 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:36.215969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-106bd807-8938-456c-a95d-256fbc4dd638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:36.285885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106bd807-8938-456c-a95d-256fbc4dd638 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:36.290886 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353b234-dae4-4d65-bf34-476a87d99f82 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbf513ee25374e49b3d8ce28843e393a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:36.291524 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353b234-dae4-4d65-bf34-476a87d99f82 service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:36.291809 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353b234-dae4-4d65-bf34-476a87d99f82 service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:36.292128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353b234-dae4-4d65-bf34-476a87d99f82 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:36.300515 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3353b234-dae4-4d65-bf34-476a87d99f82 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:36.302220 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3353b234-dae4-4d65-bf34-476a87d99f82 service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:36.310644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1133/2263] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Jul 18 14:11:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:11:43.660299 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26ef5d-ed64-4424-bf66-fb75fb2f124d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:43.662534 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ff26ef5d-ed64-4424-bf66-fb75fb2f124d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:43.781443 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26ef5d-ed64-4424-bf66-fb75fb2f124d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:43.787918 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc551b-f3f0-434f-a12d-26c31a36aa44 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd5b2ba9e37041c78a99f1700a4970c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:43.789040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc551b-f3f0-434f-a12d-26c31a36aa44 service placement] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:43.789766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc551b-f3f0-434f-a12d-26c31a36aa44 service placement] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:43.790275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc551b-f3f0-434f-a12d-26c31a36aa44 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:43.792528 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-dadc551b-f3f0-434f-a12d-26c31a36aa44 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:43.806315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc551b-f3f0-434f-a12d-26c31a36aa44 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:43.807982 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc551b-f3f0-434f-a12d-26c31a36aa44 service placement] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:43.820494 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1131/2264] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:44.256284 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353b234-dae4-4d65-bf34-476a87d99f82 service swift] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:44.363989 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353b234-dae4-4d65-bf34-476a87d99f82 service swift] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:44.370291 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e285b1-d219-4beb-a10d-5f89c6a2914c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:44.371514 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e285b1-d219-4beb-a10d-5f89c6a2914c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:44.371980 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e285b1-d219-4beb-a10d-5f89c6a2914c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:44.374384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e285b1-d219-4beb-a10d-5f89c6a2914c admin admin] PATH_INFO: `/v3/users/50650edbfbeb421592c87ff3b21ed132` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:44.376030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e285b1-d219-4beb-a10d-5f89c6a2914c admin admin] RBAC: Authorizing `identity:delete_user(user_id=50650edbfbeb421592c87ff3b21ed132)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:44.378384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e285b1-d219-4beb-a10d-5f89c6a2914c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:44.457735 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-73e285b1-d219-4beb-a10d-5f89c6a2914c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50650edbfbeb421592c87ff3b21ed132'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:44.472839 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-73e285b1-d219-4beb-a10d-5f89c6a2914c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50650edbfbeb421592c87ff3b21ed132'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:44.486823 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-73e285b1-d219-4beb-a10d-5f89c6a2914c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50650edbfbeb421592c87ff3b21ed132'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:44.487708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-73e285b1-d219-4beb-a10d-5f89c6a2914c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50650edbfbeb421592c87ff3b21ed132'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:44.514506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-73e285b1-d219-4beb-a10d-5f89c6a2914c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50650edbfbeb421592c87ff3b21ed132'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:44.533161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1134/2265] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:11:44 2022] DELETE /identity/v3/users/50650edbfbeb421592c87ff3b21ed132 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:44.538419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bd1e5-7c0b-4bcc-ad9f-8bd20457c208 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:44.538719 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bd1e5-7c0b-4bcc-ad9f-8bd20457c208 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:44.539011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bd1e5-7c0b-4bcc-ad9f-8bd20457c208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:44.558699 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-141bd1e5-7c0b-4bcc-ad9f-8bd20457c208 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:44.559928 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-141bd1e5-7c0b-4bcc-ad9f-8bd20457c208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:44.632755 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1132/2266] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:44.642358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e285b1-d219-4beb-a10d-5f89c6a2914c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:44.644649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-73e285b1-d219-4beb-a10d-5f89c6a2914c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:44.660387 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:44.753279 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e285b1-d219-4beb-a10d-5f89c6a2914c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:44.757932 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578be691-3c4a-4ee5-9f8c-907ce3f5e4e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:44.762690 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578be691-3c4a-4ee5-9f8c-907ce3f5e4e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:44.762690 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578be691-3c4a-4ee5-9f8c-907ce3f5e4e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:44.762690 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578be691-3c4a-4ee5-9f8c-907ce3f5e4e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:44.777685 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578be691-3c4a-4ee5-9f8c-907ce3f5e4e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:44.780219 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578be691-3c4a-4ee5-9f8c-907ce3f5e4e4 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:44.801349 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1135/2267] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:45.331278 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141bd1e5-7c0b-4bcc-ad9f-8bd20457c208 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:45.333064 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-141bd1e5-7c0b-4bcc-ad9f-8bd20457c208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:45.357218 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141bd1e5-7c0b-4bcc-ad9f-8bd20457c208 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:45.365361 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9616a-3436-4847-9454-ae6bdec80924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:45.366909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9616a-3436-4847-9454-ae6bdec80924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:45.367453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9616a-3436-4847-9454-ae6bdec80924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:45.368361 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9616a-3436-4847-9454-ae6bdec80924 admin admin] PATH_INFO: `/v3/projects/cfab1efff31f49b79d52a9b7cbb11b73` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:45.370300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9616a-3436-4847-9454-ae6bdec80924 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfab1efff31f49b79d52a9b7cbb11b73)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:45.372533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9616a-3436-4847-9454-ae6bdec80924 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:45.413821 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:45.438994 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-b7a9616a-3436-4847-9454-ae6bdec80924 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfab1efff31f49b79d52a9b7cbb11b73', 'request_id': 'req-b7a9616a-3436-4847-9454-ae6bdec80924'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:45.450116 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-b7a9616a-3436-4847-9454-ae6bdec80924 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfab1efff31f49b79d52a9b7cbb11b73', 'request_id': 'req-b7a9616a-3436-4847-9454-ae6bdec80924'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:45.472821 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-b7a9616a-3436-4847-9454-ae6bdec80924 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfab1efff31f49b79d52a9b7cbb11b73', 'request_id': 'req-b7a9616a-3436-4847-9454-ae6bdec80924'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:45.489619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-b7a9616a-3436-4847-9454-ae6bdec80924 admin admin] The token cache is being invalidate because project cfab1efff31f49b79d52a9b7cbb11b73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:11:45.489918 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-b7a9616a-3436-4847-9454-ae6bdec80924 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:45.490582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-b7a9616a-3436-4847-9454-ae6bdec80924 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:45.492321 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1133/2268] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:11:45 2022] DELETE /identity/v3/projects/cfab1efff31f49b79d52a9b7cbb11b73 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:45.496327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578be691-3c4a-4ee5-9f8c-907ce3f5e4e4 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:45.506492 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578be691-3c4a-4ee5-9f8c-907ce3f5e4e4 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:45.584755 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70445e59-93e2-48fa-a7a2-ebf64e65fbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:45.585646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70445e59-93e2-48fa-a7a2-ebf64e65fbc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:45.585934 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70445e59-93e2-48fa-a7a2-ebf64e65fbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:45.586210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70445e59-93e2-48fa-a7a2-ebf64e65fbc7 admin admin] PATH_INFO: `/v3/users/7ca1ce6eb8d34837b7508325909e5b48` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:45.588193 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70445e59-93e2-48fa-a7a2-ebf64e65fbc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ca1ce6eb8d34837b7508325909e5b48)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:45.591035 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70445e59-93e2-48fa-a7a2-ebf64e65fbc7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:45.653255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-70445e59-93e2-48fa-a7a2-ebf64e65fbc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ca1ce6eb8d34837b7508325909e5b48'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:45.665457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-70445e59-93e2-48fa-a7a2-ebf64e65fbc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ca1ce6eb8d34837b7508325909e5b48'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:45.675122 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-70445e59-93e2-48fa-a7a2-ebf64e65fbc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ca1ce6eb8d34837b7508325909e5b48'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:45.676161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-70445e59-93e2-48fa-a7a2-ebf64e65fbc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ca1ce6eb8d34837b7508325909e5b48'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:45.690982 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-70445e59-93e2-48fa-a7a2-ebf64e65fbc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ca1ce6eb8d34837b7508325909e5b48'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:45.701389 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1136/2269] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:11:45 2022] DELETE /identity/v3/users/7ca1ce6eb8d34837b7508325909e5b48 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:46.068741 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9616a-3436-4847-9454-ae6bdec80924 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:46.071265 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b7a9616a-3436-4847-9454-ae6bdec80924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:46.096357 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9616a-3436-4847-9454-ae6bdec80924 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:46.182513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c5ae0e-c395-456f-a689-e40c5d6c1753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:46.184857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5ae0e-c395-456f-a689-e40c5d6c1753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:46.185307 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5ae0e-c395-456f-a689-e40c5d6c1753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:46.185604 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5ae0e-c395-456f-a689-e40c5d6c1753 admin admin] PATH_INFO: `/v3/projects/933725adaa1741e98ab4d4f50c739616` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:46.187371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c5ae0e-c395-456f-a689-e40c5d6c1753 admin admin] RBAC: Authorizing `identity:delete_project(project_id=933725adaa1741e98ab4d4f50c739616)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:46.189241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c5ae0e-c395-456f-a689-e40c5d6c1753 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:46.249394 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-47c5ae0e-c395-456f-a689-e40c5d6c1753 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '933725adaa1741e98ab4d4f50c739616', 'request_id': 'req-47c5ae0e-c395-456f-a689-e40c5d6c1753'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:46.257155 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-47c5ae0e-c395-456f-a689-e40c5d6c1753 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '933725adaa1741e98ab4d4f50c739616', 'request_id': 'req-47c5ae0e-c395-456f-a689-e40c5d6c1753'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:46.268673 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-47c5ae0e-c395-456f-a689-e40c5d6c1753 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '933725adaa1741e98ab4d4f50c739616', 'request_id': 'req-47c5ae0e-c395-456f-a689-e40c5d6c1753'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:46.282925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-47c5ae0e-c395-456f-a689-e40c5d6c1753 admin admin] The token cache is being invalidate because project 933725adaa1741e98ab4d4f50c739616 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:11:46.283226 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-47c5ae0e-c395-456f-a689-e40c5d6c1753 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:46.283992 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-47c5ae0e-c395-456f-a689-e40c5d6c1753 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:46.285791 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1134/2270] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:11:46 2022] DELETE /identity/v3/projects/933725adaa1741e98ab4d4f50c739616 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:46.290635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac38fb7-56c5-4f54-8a9f-c0da6e591057 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:46.291097 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac38fb7-56c5-4f54-8a9f-c0da6e591057 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:46.291482 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac38fb7-56c5-4f54-8a9f-c0da6e591057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:46.313365 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8ac38fb7-56c5-4f54-8a9f-c0da6e591057 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:46.314585 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8ac38fb7-56c5-4f54-8a9f-c0da6e591057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:46.346751 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:46.395296 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1137/2271] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:46.404036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb541ecf-65d8-4423-abb6-4259636c817a None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:46.405103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb541ecf-65d8-4423-abb6-4259636c817a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:46.405806 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb541ecf-65d8-4423-abb6-4259636c817a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:46.435319 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-eb541ecf-65d8-4423-abb6-4259636c817a None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:46.436838 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-eb541ecf-65d8-4423-abb6-4259636c817a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:46.447862 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1135/2272] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:46.452123 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac38fb7-56c5-4f54-8a9f-c0da6e591057 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:46.469578 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac38fb7-56c5-4f54-8a9f-c0da6e591057 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:46.475458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb79c5-711c-41e5-a6a2-55c3e6802393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:46.476336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb79c5-711c-41e5-a6a2-55c3e6802393 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:46.476628 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb79c5-711c-41e5-a6a2-55c3e6802393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:46.476919 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb79c5-711c-41e5-a6a2-55c3e6802393 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:46.477687 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb79c5-711c-41e5-a6a2-55c3e6802393 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:11:46.478123 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb79c5-711c-41e5-a6a2-55c3e6802393 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:46.479789 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb79c5-711c-41e5-a6a2-55c3e6802393 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:46.507175 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1138/2273] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:11:46.508470 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb541ecf-65d8-4423-abb6-4259636c817a None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:46.520661 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb541ecf-65d8-4423-abb6-4259636c817a None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:46.528356 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663cfcc-59b7-4105-8cda-68f313da9a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:46.529112 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663cfcc-59b7-4105-8cda-68f313da9a86 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:46.529481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663cfcc-59b7-4105-8cda-68f313da9a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:46.529841 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663cfcc-59b7-4105-8cda-68f313da9a86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:46.530726 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a663cfcc-59b7-4105-8cda-68f313da9a86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:46.532807 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a663cfcc-59b7-4105-8cda-68f313da9a86 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:46.566813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1136/2274] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:11:46 2022] POST /identity/v3/projects => generated 387 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:11:46.571096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb79c5-711c-41e5-a6a2-55c3e6802393 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:46.582771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb79c5-711c-41e5-a6a2-55c3e6802393 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:46.660893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06500a3b-7cca-4b25-9556-6897913f8360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:46.661563 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06500a3b-7cca-4b25-9556-6897913f8360 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:46.662044 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06500a3b-7cca-4b25-9556-6897913f8360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:46.662481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06500a3b-7cca-4b25-9556-6897913f8360 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:46.663337 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06500a3b-7cca-4b25-9556-6897913f8360 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:46.665453 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06500a3b-7cca-4b25-9556-6897913f8360 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:46.692849 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1139/2275] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:11:46 2022] POST /identity/v3/users => generated 294 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:11:46.694063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663cfcc-59b7-4105-8cda-68f313da9a86 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:46.704604 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663cfcc-59b7-4105-8cda-68f313da9a86 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:46.709923 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62141b2-f5ce-4b4a-b765-d6339b364c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:46.710781 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62141b2-f5ce-4b4a-b765-d6339b364c16 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:46.711101 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62141b2-f5ce-4b4a-b765-d6339b364c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:46.711348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62141b2-f5ce-4b4a-b765-d6339b364c16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:46.712187 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62141b2-f5ce-4b4a-b765-d6339b364c16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:46.713991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62141b2-f5ce-4b4a-b765-d6339b364c16 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:46.729853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1137/2276] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:11:46.730912 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06500a3b-7cca-4b25-9556-6897913f8360 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:46.744736 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06500a3b-7cca-4b25-9556-6897913f8360 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:46.754525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af590f-a6cd-4067-af8a-0a6d0b0ffea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:46.755568 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af590f-a6cd-4067-af8a-0a6d0b0ffea0 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:46.756460 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af590f-a6cd-4067-af8a-0a6d0b0ffea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:46.757344 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af590f-a6cd-4067-af8a-0a6d0b0ffea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:46.760768 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af590f-a6cd-4067-af8a-0a6d0b0ffea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:46.763425 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af590f-a6cd-4067-af8a-0a6d0b0ffea0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:46.780490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1140/2277] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:11:46.785472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62141b2-f5ce-4b4a-b765-d6339b364c16 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:46.798284 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62141b2-f5ce-4b4a-b765-d6339b364c16 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:46.811015 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce491c6-0e07-4966-8daf-37f8e83a2cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:46.812827 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce491c6-0e07-4966-8daf-37f8e83a2cd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:46.813536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce491c6-0e07-4966-8daf-37f8e83a2cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:46.814227 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce491c6-0e07-4966-8daf-37f8e83a2cd5 admin admin] PATH_INFO: `/v3/projects/d06b4d82aa7840ca84786dc203d90980/users/023547192f5e400eac2a314ea7e7b973/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:46.820441 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:46.875285 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce491c6-0e07-4966-8daf-37f8e83a2cd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d06b4d82aa7840ca84786dc203d90980, user_id=023547192f5e400eac2a314ea7e7b973, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:46.881494 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce491c6-0e07-4966-8daf-37f8e83a2cd5 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:46.909520 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1138/2278] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:11:46 2022] PUT /identity/v3/projects/d06b4d82aa7840ca84786dc203d90980/users/023547192f5e400eac2a314ea7e7b973/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:11:46.919598 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000188a1-86a6-4a57-a612-dd22be83386d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:46.920235 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000188a1-86a6-4a57-a612-dd22be83386d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:46.920680 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000188a1-86a6-4a57-a612-dd22be83386d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:46.988628 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-000188a1-86a6-4a57-a612-dd22be83386d None None] MFA Rules not processed for user `023547192f5e400eac2a314ea7e7b973`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:46.990301 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-000188a1-86a6-4a57-a612-dd22be83386d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:47.162003 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1141/2279] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:47.166989 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce491c6-0e07-4966-8daf-37f8e83a2cd5 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:47.324540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce491c6-0e07-4966-8daf-37f8e83a2cd5 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:47.334265 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323af13-04bd-46e0-ac15-7b44233ee274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:47.335447 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323af13-04bd-46e0-ac15-7b44233ee274 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:47.336106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323af13-04bd-46e0-ac15-7b44233ee274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:47.337111 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323af13-04bd-46e0-ac15-7b44233ee274 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:47.338660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3323af13-04bd-46e0-ac15-7b44233ee274 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:47.342444 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3323af13-04bd-46e0-ac15-7b44233ee274 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:47.383891 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1139/2280] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:11:47 2022] POST /identity/v3/projects => generated 387 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:11:47.390456 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000188a1-86a6-4a57-a612-dd22be83386d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:47.404316 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000188a1-86a6-4a57-a612-dd22be83386d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:47.430752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:47.468284 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c516c1-8132-47c8-bdab-0aa449933d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:47.469103 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c516c1-8132-47c8-bdab-0aa449933d1c admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:47.469599 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c516c1-8132-47c8-bdab-0aa449933d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:47.469971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c516c1-8132-47c8-bdab-0aa449933d1c admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:47.470873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c516c1-8132-47c8-bdab-0aa449933d1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:47.473514 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c516c1-8132-47c8-bdab-0aa449933d1c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:47.498830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1142/2281] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:11:47 2022] POST /identity/v3/users => generated 300 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:11:47.502746 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323af13-04bd-46e0-ac15-7b44233ee274 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:47.514992 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323af13-04bd-46e0-ac15-7b44233ee274 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:47.522366 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b6980-a2ce-4c10-a2f9-9336a6b24651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:47.523049 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b6980-a2ce-4c10-a2f9-9336a6b24651 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:47.523344 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b6980-a2ce-4c10-a2f9-9336a6b24651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:47.523626 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b6980-a2ce-4c10-a2f9-9336a6b24651 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:47.524406 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b6980-a2ce-4c10-a2f9-9336a6b24651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:47.526175 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b6980-a2ce-4c10-a2f9-9336a6b24651 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:47.539602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1140/2282] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:11:47.543578 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c516c1-8132-47c8-bdab-0aa449933d1c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:47.556594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c516c1-8132-47c8-bdab-0aa449933d1c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:47.562269 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1e2e4-ec86-42d5-9a37-29fee3effbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:47.563709 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1e2e4-ec86-42d5-9a37-29fee3effbd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:47.564117 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1e2e4-ec86-42d5-9a37-29fee3effbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:47.564470 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1e2e4-ec86-42d5-9a37-29fee3effbd4 admin admin] PATH_INFO: `/v3/projects/072f21611d554344b0c797ba31554029/users/3570ac62945e4857aca6fa07593f916e/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:47.582088 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c1e2e4-ec86-42d5-9a37-29fee3effbd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=072f21611d554344b0c797ba31554029, user_id=3570ac62945e4857aca6fa07593f916e, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:47.585159 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c1e2e4-ec86-42d5-9a37-29fee3effbd4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:47.603757 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1143/2283] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:11:47 2022] PUT /identity/v3/projects/072f21611d554344b0c797ba31554029/users/3570ac62945e4857aca6fa07593f916e/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:47.607724 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b6980-a2ce-4c10-a2f9-9336a6b24651 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:47.625316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b6980-a2ce-4c10-a2f9-9336a6b24651 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:47.695069 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f12f8-aa3a-4d4f-8af1-eba4569bb379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:47.695822 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f12f8-aa3a-4d4f-8af1-eba4569bb379 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:47.696455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f12f8-aa3a-4d4f-8af1-eba4569bb379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:47.696738 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f12f8-aa3a-4d4f-8af1-eba4569bb379 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:47.697489 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f12f8-aa3a-4d4f-8af1-eba4569bb379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:47.699195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f12f8-aa3a-4d4f-8af1-eba4569bb379 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:47.710831 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1141/2284] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:11:47.714967 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1e2e4-ec86-42d5-9a37-29fee3effbd4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:47.726048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1e2e4-ec86-42d5-9a37-29fee3effbd4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:47.731831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f48db4-b7c8-4fd0-a0f4-7ba77741437c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:47.733001 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f48db4-b7c8-4fd0-a0f4-7ba77741437c admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:47.733287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f48db4-b7c8-4fd0-a0f4-7ba77741437c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:47.733596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f48db4-b7c8-4fd0-a0f4-7ba77741437c admin admin] PATH_INFO: `/v3/projects/072f21611d554344b0c797ba31554029/users/3570ac62945e4857aca6fa07593f916e/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:47.735565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f48db4-b7c8-4fd0-a0f4-7ba77741437c admin admin] RBAC: Authorizing `identity:create_grant(project_id=072f21611d554344b0c797ba31554029, user_id=3570ac62945e4857aca6fa07593f916e, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:47.738504 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f48db4-b7c8-4fd0-a0f4-7ba77741437c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:47.751244 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1144/2285] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:11:47 2022] PUT /identity/v3/projects/072f21611d554344b0c797ba31554029/users/3570ac62945e4857aca6fa07593f916e/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:47.762062 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b37d79-f340-43fb-b4d6-140f8acf342e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:47.762480 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b37d79-f340-43fb-b4d6-140f8acf342e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:47.762866 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b37d79-f340-43fb-b4d6-140f8acf342e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:47.808376 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-55b37d79-f340-43fb-b4d6-140f8acf342e None None] MFA Rules not processed for user `3570ac62945e4857aca6fa07593f916e`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:47.809713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-55b37d79-f340-43fb-b4d6-140f8acf342e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:47.867887 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:47.905462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1142/2286] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:47 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:47.917146 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f48db4-b7c8-4fd0-a0f4-7ba77741437c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:47.918943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-52f48db4-b7c8-4fd0-a0f4-7ba77741437c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:47.982941 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f48db4-b7c8-4fd0-a0f4-7ba77741437c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:47.987058 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2d994-cc81-471d-b85c-026b77df70c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:47.987861 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2d994-cc81-471d-b85c-026b77df70c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:47.988356 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2d994-cc81-471d-b85c-026b77df70c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:47.988608 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2d994-cc81-471d-b85c-026b77df70c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:47.999475 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2d994-cc81-471d-b85c-026b77df70c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:48.001112 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2d994-cc81-471d-b85c-026b77df70c2 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:48.035465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1145/2287] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:47 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:48.087337 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b37d79-f340-43fb-b4d6-140f8acf342e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:48.182318 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b37d79-f340-43fb-b4d6-140f8acf342e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:48.187636 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bec28e-b8a1-4d82-8dfb-5e11e9d6eae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:48.189202 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bec28e-b8a1-4d82-8dfb-5e11e9d6eae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:48.189526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bec28e-b8a1-4d82-8dfb-5e11e9d6eae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:48.189846 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bec28e-b8a1-4d82-8dfb-5e11e9d6eae0 admin admin] PATH_INFO: `/v3/users/023547192f5e400eac2a314ea7e7b973` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:48.191650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bec28e-b8a1-4d82-8dfb-5e11e9d6eae0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=023547192f5e400eac2a314ea7e7b973)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:48.193674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bec28e-b8a1-4d82-8dfb-5e11e9d6eae0 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:48.253792 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-79bec28e-b8a1-4d82-8dfb-5e11e9d6eae0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '023547192f5e400eac2a314ea7e7b973'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:48.261078 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-79bec28e-b8a1-4d82-8dfb-5e11e9d6eae0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '023547192f5e400eac2a314ea7e7b973'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:48.270356 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-79bec28e-b8a1-4d82-8dfb-5e11e9d6eae0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '023547192f5e400eac2a314ea7e7b973'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:48.271024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-79bec28e-b8a1-4d82-8dfb-5e11e9d6eae0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '023547192f5e400eac2a314ea7e7b973'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:48.284498 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-79bec28e-b8a1-4d82-8dfb-5e11e9d6eae0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '023547192f5e400eac2a314ea7e7b973'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:48.292158 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1143/2288] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:11:48 2022] DELETE /identity/v3/users/023547192f5e400eac2a314ea7e7b973 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:48.296391 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3210b15-bd27-49f0-af32-a228cba1fe31 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:48.296685 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3210b15-bd27-49f0-af32-a228cba1fe31 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:48.296969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3210b15-bd27-49f0-af32-a228cba1fe31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:48.321012 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c3210b15-bd27-49f0-af32-a228cba1fe31 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:48.322919 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c3210b15-bd27-49f0-af32-a228cba1fe31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:48.392672 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1146/2289] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:11:48.403143 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bec28e-b8a1-4d82-8dfb-5e11e9d6eae0 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:48.405092 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-79bec28e-b8a1-4d82-8dfb-5e11e9d6eae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:48.479339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bec28e-b8a1-4d82-8dfb-5e11e9d6eae0 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:48.483628 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb8ccd-92eb-463f-bda7-963acc8b08c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:48.484625 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb8ccd-92eb-463f-bda7-963acc8b08c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:48.485144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb8ccd-92eb-463f-bda7-963acc8b08c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:48.485640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb8ccd-92eb-463f-bda7-963acc8b08c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:48.497014 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb8ccd-92eb-463f-bda7-963acc8b08c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:48.498521 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb8ccd-92eb-463f-bda7-963acc8b08c2 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:48.510491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1144/2290] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:48.994575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3210b15-bd27-49f0-af32-a228cba1fe31 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:48.997404 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c3210b15-bd27-49f0-af32-a228cba1fe31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:49.017426 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3210b15-bd27-49f0-af32-a228cba1fe31 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:49.029076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018649c1-83f7-43d2-9d10-fece47c89007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:49.030185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018649c1-83f7-43d2-9d10-fece47c89007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:49.030629 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018649c1-83f7-43d2-9d10-fece47c89007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:49.031079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018649c1-83f7-43d2-9d10-fece47c89007 admin admin] PATH_INFO: `/v3/projects/d06b4d82aa7840ca84786dc203d90980` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:49.032976 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018649c1-83f7-43d2-9d10-fece47c89007 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d06b4d82aa7840ca84786dc203d90980)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:49.036476 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018649c1-83f7-43d2-9d10-fece47c89007 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:49.058857 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:49.104323 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-018649c1-83f7-43d2-9d10-fece47c89007 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd06b4d82aa7840ca84786dc203d90980', 'request_id': 'req-018649c1-83f7-43d2-9d10-fece47c89007'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:49.121270 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-018649c1-83f7-43d2-9d10-fece47c89007 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd06b4d82aa7840ca84786dc203d90980', 'request_id': 'req-018649c1-83f7-43d2-9d10-fece47c89007'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:49.131764 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-018649c1-83f7-43d2-9d10-fece47c89007 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd06b4d82aa7840ca84786dc203d90980', 'request_id': 'req-018649c1-83f7-43d2-9d10-fece47c89007'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:49.146367 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-018649c1-83f7-43d2-9d10-fece47c89007 admin admin] The token cache is being invalidate because project d06b4d82aa7840ca84786dc203d90980 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:11:49.146659 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-018649c1-83f7-43d2-9d10-fece47c89007 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:49.147301 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-018649c1-83f7-43d2-9d10-fece47c89007 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:49.149322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1147/2291] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:11:48 2022] DELETE /identity/v3/projects/d06b4d82aa7840ca84786dc203d90980 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:49.153875 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb8ccd-92eb-463f-bda7-963acc8b08c2 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:49.159459 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:49.166376 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb8ccd-92eb-463f-bda7-963acc8b08c2 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:49.236222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a728737-342e-4ba0-9f17-42d9fd438e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:49.237150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a728737-342e-4ba0-9f17-42d9fd438e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:49.237504 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a728737-342e-4ba0-9f17-42d9fd438e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:49.237784 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a728737-342e-4ba0-9f17-42d9fd438e17 admin admin] PATH_INFO: `/v3/users/3570ac62945e4857aca6fa07593f916e` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:49.239009 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a728737-342e-4ba0-9f17-42d9fd438e17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3570ac62945e4857aca6fa07593f916e)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:49.240976 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a728737-342e-4ba0-9f17-42d9fd438e17 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:49.317410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0a728737-342e-4ba0-9f17-42d9fd438e17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3570ac62945e4857aca6fa07593f916e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:49.333136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0a728737-342e-4ba0-9f17-42d9fd438e17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3570ac62945e4857aca6fa07593f916e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:49.343295 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0a728737-342e-4ba0-9f17-42d9fd438e17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3570ac62945e4857aca6fa07593f916e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:49.344973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0a728737-342e-4ba0-9f17-42d9fd438e17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3570ac62945e4857aca6fa07593f916e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:49.361219 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0a728737-342e-4ba0-9f17-42d9fd438e17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3570ac62945e4857aca6fa07593f916e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:49.375390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1145/2292] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:11:49 2022] DELETE /identity/v3/users/3570ac62945e4857aca6fa07593f916e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:49.707976 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018649c1-83f7-43d2-9d10-fece47c89007 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:49.709502 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-018649c1-83f7-43d2-9d10-fece47c89007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:49.724364 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018649c1-83f7-43d2-9d10-fece47c89007 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:49.808800 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22cd7b8-2cf8-403a-9116-7403f7d0cd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:49.809724 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cd7b8-2cf8-403a-9116-7403f7d0cd80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:49.810124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cd7b8-2cf8-403a-9116-7403f7d0cd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:49.810404 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cd7b8-2cf8-403a-9116-7403f7d0cd80 admin admin] PATH_INFO: `/v3/projects/072f21611d554344b0c797ba31554029` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:49.812791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22cd7b8-2cf8-403a-9116-7403f7d0cd80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=072f21611d554344b0c797ba31554029)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:49.816537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22cd7b8-2cf8-403a-9116-7403f7d0cd80 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:49.875996 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f22cd7b8-2cf8-403a-9116-7403f7d0cd80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '072f21611d554344b0c797ba31554029', 'request_id': 'req-f22cd7b8-2cf8-403a-9116-7403f7d0cd80'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:49.882873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f22cd7b8-2cf8-403a-9116-7403f7d0cd80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '072f21611d554344b0c797ba31554029', 'request_id': 'req-f22cd7b8-2cf8-403a-9116-7403f7d0cd80'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:49.891826 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f22cd7b8-2cf8-403a-9116-7403f7d0cd80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '072f21611d554344b0c797ba31554029', 'request_id': 'req-f22cd7b8-2cf8-403a-9116-7403f7d0cd80'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:49.902992 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f22cd7b8-2cf8-403a-9116-7403f7d0cd80 admin admin] The token cache is being invalidate because project 072f21611d554344b0c797ba31554029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:11:49.903303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f22cd7b8-2cf8-403a-9116-7403f7d0cd80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:49.903924 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f22cd7b8-2cf8-403a-9116-7403f7d0cd80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:49.905556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1148/2293] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:11:49 2022] DELETE /identity/v3/projects/072f21611d554344b0c797ba31554029 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:49.911104 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5e59b-5245-49d4-aac4-99f27a8646b5 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:49.911641 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5e59b-5245-49d4-aac4-99f27a8646b5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:49.911825 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5e59b-5245-49d4-aac4-99f27a8646b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:49.931773 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d0a5e59b-5245-49d4-aac4-99f27a8646b5 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:49.933152 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d0a5e59b-5245-49d4-aac4-99f27a8646b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:50.016161 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1146/2294] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:50.017766 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858960cf-4192-4b28-89d6-dc92a1bb3255 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:50.018286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858960cf-4192-4b28-89d6-dc92a1bb3255 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:50.018625 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858960cf-4192-4b28-89d6-dc92a1bb3255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:50.038882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-858960cf-4192-4b28-89d6-dc92a1bb3255 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:50.040315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-858960cf-4192-4b28-89d6-dc92a1bb3255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:50.049140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1149/2295] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:50.054062 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5e59b-5245-49d4-aac4-99f27a8646b5 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:50.064569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5e59b-5245-49d4-aac4-99f27a8646b5 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:50.070547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bc4dd-be12-4178-86cf-fcceacdea7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:50.071309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bc4dd-be12-4178-86cf-fcceacdea7cd admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:50.071692 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bc4dd-be12-4178-86cf-fcceacdea7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:50.072049 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bc4dd-be12-4178-86cf-fcceacdea7cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:50.072867 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532bc4dd-be12-4178-86cf-fcceacdea7cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:11:50.073247 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532bc4dd-be12-4178-86cf-fcceacdea7cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:50.074925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532bc4dd-be12-4178-86cf-fcceacdea7cd admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:50.092763 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858960cf-4192-4b28-89d6-dc92a1bb3255 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:50.103339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1147/2296] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:11:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:11:50.103927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858960cf-4192-4b28-89d6-dc92a1bb3255 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:50.110761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d22b6-9fb4-41b0-8c29-99db82504d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:50.114680 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d22b6-9fb4-41b0-8c29-99db82504d15 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:50.114680 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d22b6-9fb4-41b0-8c29-99db82504d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:50.115550 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d22b6-9fb4-41b0-8c29-99db82504d15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:50.115841 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d22b6-9fb4-41b0-8c29-99db82504d15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:50.117879 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d22b6-9fb4-41b0-8c29-99db82504d15 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:50.154563 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1150/2297] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:11:50 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:11:50.157930 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bc4dd-be12-4178-86cf-fcceacdea7cd admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:50.173964 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bc4dd-be12-4178-86cf-fcceacdea7cd admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:50.179045 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:50.269211 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9fd8e-834d-4584-a6ff-2fc08a46bd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:50.269991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9fd8e-834d-4584-a6ff-2fc08a46bd34 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:50.270361 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9fd8e-834d-4584-a6ff-2fc08a46bd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:50.270715 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9fd8e-834d-4584-a6ff-2fc08a46bd34 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:50.271492 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9fd8e-834d-4584-a6ff-2fc08a46bd34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:50.273669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9fd8e-834d-4584-a6ff-2fc08a46bd34 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:50.308891 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1148/2298] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:11:50 2022] POST /identity/v3/users => generated 293 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:11:50.314078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d22b6-9fb4-41b0-8c29-99db82504d15 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:50.337347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d22b6-9fb4-41b0-8c29-99db82504d15 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:50.345480 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b985c3-f71f-482e-bbc3-2f9fd0ceb485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:50.346402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b985c3-f71f-482e-bbc3-2f9fd0ceb485 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:50.346795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b985c3-f71f-482e-bbc3-2f9fd0ceb485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:50.347207 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b985c3-f71f-482e-bbc3-2f9fd0ceb485 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:50.348169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b985c3-f71f-482e-bbc3-2f9fd0ceb485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:50.350475 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b985c3-f71f-482e-bbc3-2f9fd0ceb485 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:50.352306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:50.374055 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1151/2299] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:11:50.375351 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9fd8e-834d-4584-a6ff-2fc08a46bd34 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:50.388652 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9fd8e-834d-4584-a6ff-2fc08a46bd34 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:50.394613 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2c0d1-3f9b-42f9-9e77-37a96b7666b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:50.395657 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2c0d1-3f9b-42f9-9e77-37a96b7666b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:50.396185 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2c0d1-3f9b-42f9-9e77-37a96b7666b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:50.396498 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2c0d1-3f9b-42f9-9e77-37a96b7666b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:50.397310 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c2c0d1-3f9b-42f9-9e77-37a96b7666b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:50.399049 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c2c0d1-3f9b-42f9-9e77-37a96b7666b8 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:50.411358 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1149/2300] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:11:50.415872 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b985c3-f71f-482e-bbc3-2f9fd0ceb485 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:50.427294 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b985c3-f71f-482e-bbc3-2f9fd0ceb485 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:50.434032 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15866a87-d478-42bc-aaba-8f469ce59cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:50.434032 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15866a87-d478-42bc-aaba-8f469ce59cac admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:50.434032 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15866a87-d478-42bc-aaba-8f469ce59cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:50.434032 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15866a87-d478-42bc-aaba-8f469ce59cac admin admin] PATH_INFO: `/v3/projects/8b5e20dc231945c89a98821684f9ed2f/users/a4b29d7ef7044d249ef9c6025dfd4a67/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:50.452029 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15866a87-d478-42bc-aaba-8f469ce59cac admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b5e20dc231945c89a98821684f9ed2f, user_id=a4b29d7ef7044d249ef9c6025dfd4a67, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:50.454883 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15866a87-d478-42bc-aaba-8f469ce59cac admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:50.470519 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1152/2301] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:11:50 2022] PUT /identity/v3/projects/8b5e20dc231945c89a98821684f9ed2f/users/a4b29d7ef7044d249ef9c6025dfd4a67/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:50.481971 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861d912-88b4-4cd5-a151-758670163e9f None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:50.482578 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861d912-88b4-4cd5-a151-758670163e9f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:50.483053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861d912-88b4-4cd5-a151-758670163e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:50.518832 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-5861d912-88b4-4cd5-a151-758670163e9f None None] MFA Rules not processed for user `a4b29d7ef7044d249ef9c6025dfd4a67`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:50.520194 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5861d912-88b4-4cd5-a151-758670163e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:50.633892 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1150/2302] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:50.635196 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15866a87-d478-42bc-aaba-8f469ce59cac admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:50.718423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15866a87-d478-42bc-aaba-8f469ce59cac admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:50.724217 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391a97f-76d9-4960-9ffb-c64c2d011b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:50.725236 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391a97f-76d9-4960-9ffb-c64c2d011b6c admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:50.726027 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391a97f-76d9-4960-9ffb-c64c2d011b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:50.726715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391a97f-76d9-4960-9ffb-c64c2d011b6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:50.727713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6391a97f-76d9-4960-9ffb-c64c2d011b6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:50.730259 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6391a97f-76d9-4960-9ffb-c64c2d011b6c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:50.760280 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1153/2303] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:11:50 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:11:50.764441 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861d912-88b4-4cd5-a151-758670163e9f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:50.781096 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861d912-88b4-4cd5-a151-758670163e9f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:50.860701 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ddfed-2e6d-4e40-ad75-6ad1bebf41e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:50.861541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ddfed-2e6d-4e40-ad75-6ad1bebf41e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:50.862463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ddfed-2e6d-4e40-ad75-6ad1bebf41e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:50.862852 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ddfed-2e6d-4e40-ad75-6ad1bebf41e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:50.863651 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ddfed-2e6d-4e40-ad75-6ad1bebf41e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:50.865822 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ddfed-2e6d-4e40-ad75-6ad1bebf41e8 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:50.890291 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1151/2304] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:11:50 2022] POST /identity/v3/users => generated 299 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:11:50.894900 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391a97f-76d9-4960-9ffb-c64c2d011b6c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:50.917346 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391a97f-76d9-4960-9ffb-c64c2d011b6c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:50.923706 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798d47f-cb12-44bd-bdfd-859a2c825f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:50.925302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798d47f-cb12-44bd-bdfd-859a2c825f08 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:50.925708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798d47f-cb12-44bd-bdfd-859a2c825f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:50.925988 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6798d47f-cb12-44bd-bdfd-859a2c825f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:50.927072 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6798d47f-cb12-44bd-bdfd-859a2c825f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:50.929064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6798d47f-cb12-44bd-bdfd-859a2c825f08 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:50.952100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1154/2305] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:11:50.953266 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ddfed-2e6d-4e40-ad75-6ad1bebf41e8 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:50.965134 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ddfed-2e6d-4e40-ad75-6ad1bebf41e8 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:50.972469 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99569d-424e-4419-9f10-2842679069fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:50.973958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99569d-424e-4419-9f10-2842679069fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:50.974609 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99569d-424e-4419-9f10-2842679069fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:50.975135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99569d-424e-4419-9f10-2842679069fd admin admin] PATH_INFO: `/v3/projects/5231d296ede54f66ad6122fcf729b6b3/users/eb06a32d7fa44cbe83b9ac83ef9c40c0/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:50.978653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:50.999120 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99569d-424e-4419-9f10-2842679069fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5231d296ede54f66ad6122fcf729b6b3, user_id=eb06a32d7fa44cbe83b9ac83ef9c40c0, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:51.002974 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99569d-424e-4419-9f10-2842679069fd admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:51.021579 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1152/2306] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:11:50 2022] PUT /identity/v3/projects/5231d296ede54f66ad6122fcf729b6b3/users/eb06a32d7fa44cbe83b9ac83ef9c40c0/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:11:51.026485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798d47f-cb12-44bd-bdfd-859a2c825f08 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:51.039338 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6798d47f-cb12-44bd-bdfd-859a2c825f08 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:51.106143 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01dc476-6d74-4a2c-9655-902d88f5cbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:51.107212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01dc476-6d74-4a2c-9655-902d88f5cbc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:51.107554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01dc476-6d74-4a2c-9655-902d88f5cbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:51.107835 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01dc476-6d74-4a2c-9655-902d88f5cbc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:51.108681 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01dc476-6d74-4a2c-9655-902d88f5cbc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:51.110445 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01dc476-6d74-4a2c-9655-902d88f5cbc7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:51.124049 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1155/2307] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:11:51.125798 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99569d-424e-4419-9f10-2842679069fd admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:51.137385 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99569d-424e-4419-9f10-2842679069fd admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:51.142148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921739f-65c8-49c8-9ec3-8d10eb33938c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:51.143560 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921739f-65c8-49c8-9ec3-8d10eb33938c admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:51.143845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921739f-65c8-49c8-9ec3-8d10eb33938c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:51.144157 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921739f-65c8-49c8-9ec3-8d10eb33938c admin admin] PATH_INFO: `/v3/projects/5231d296ede54f66ad6122fcf729b6b3/users/eb06a32d7fa44cbe83b9ac83ef9c40c0/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:51.146161 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921739f-65c8-49c8-9ec3-8d10eb33938c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5231d296ede54f66ad6122fcf729b6b3, user_id=eb06a32d7fa44cbe83b9ac83ef9c40c0, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:51.148962 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921739f-65c8-49c8-9ec3-8d10eb33938c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:51.162100 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1153/2308] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:11:51 2022] PUT /identity/v3/projects/5231d296ede54f66ad6122fcf729b6b3/users/eb06a32d7fa44cbe83b9ac83ef9c40c0/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:51.177983 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d911578-7ee7-4c02-b27c-4219d2b2f858 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:51.178261 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d911578-7ee7-4c02-b27c-4219d2b2f858 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:51.178456 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d911578-7ee7-4c02-b27c-4219d2b2f858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:51.214675 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0d911578-7ee7-4c02-b27c-4219d2b2f858 None None] MFA Rules not processed for user `eb06a32d7fa44cbe83b9ac83ef9c40c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:51.215898 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0d911578-7ee7-4c02-b27c-4219d2b2f858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:51.252001 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:51.301487 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1156/2309] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:51.309213 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921739f-65c8-49c8-9ec3-8d10eb33938c admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:51.311485 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a921739f-65c8-49c8-9ec3-8d10eb33938c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:51.383601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921739f-65c8-49c8-9ec3-8d10eb33938c admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:51.387683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41141596-d9bb-4903-824c-311c6df7f0a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:51.388431 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41141596-d9bb-4903-824c-311c6df7f0a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:51.388803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41141596-d9bb-4903-824c-311c6df7f0a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:51.389103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41141596-d9bb-4903-824c-311c6df7f0a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:51.401519 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41141596-d9bb-4903-824c-311c6df7f0a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:51.403421 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41141596-d9bb-4903-824c-311c6df7f0a8 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:51.421239 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1154/2310] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:51.494195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d911578-7ee7-4c02-b27c-4219d2b2f858 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:51.567545 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d911578-7ee7-4c02-b27c-4219d2b2f858 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:51.573758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0c3518-7c22-45e4-a244-c93bb6c893e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:51.574888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c3518-7c22-45e4-a244-c93bb6c893e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:51.575181 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c3518-7c22-45e4-a244-c93bb6c893e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:51.575463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c3518-7c22-45e4-a244-c93bb6c893e2 admin admin] PATH_INFO: `/v3/users/a4b29d7ef7044d249ef9c6025dfd4a67` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:51.576722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0c3518-7c22-45e4-a244-c93bb6c893e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4b29d7ef7044d249ef9c6025dfd4a67)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:51.578667 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0c3518-7c22-45e4-a244-c93bb6c893e2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:51.635031 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bc0c3518-7c22-45e4-a244-c93bb6c893e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4b29d7ef7044d249ef9c6025dfd4a67'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:51.645267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bc0c3518-7c22-45e4-a244-c93bb6c893e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4b29d7ef7044d249ef9c6025dfd4a67'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:51.655829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bc0c3518-7c22-45e4-a244-c93bb6c893e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4b29d7ef7044d249ef9c6025dfd4a67'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:51.656627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bc0c3518-7c22-45e4-a244-c93bb6c893e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4b29d7ef7044d249ef9c6025dfd4a67'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:51.672492 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-bc0c3518-7c22-45e4-a244-c93bb6c893e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4b29d7ef7044d249ef9c6025dfd4a67'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:51.679113 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1157/2311] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:11:51 2022] DELETE /identity/v3/users/a4b29d7ef7044d249ef9c6025dfd4a67 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:51.686099 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d0d36-e6a8-43c0-9209-1959920b41de None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:51.686398 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d0d36-e6a8-43c0-9209-1959920b41de None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:51.686687 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d0d36-e6a8-43c0-9209-1959920b41de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:51.707014 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-bb8d0d36-e6a8-43c0-9209-1959920b41de None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:51.708300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bb8d0d36-e6a8-43c0-9209-1959920b41de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:51.783016 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1155/2312] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:51.804493 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0c3518-7c22-45e4-a244-c93bb6c893e2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:51.806404 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bc0c3518-7c22-45e4-a244-c93bb6c893e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:51.900234 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:51.909758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0c3518-7c22-45e4-a244-c93bb6c893e2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:51.943820 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78460f04-e538-413b-89fb-3cf5cc7c0286 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:51.944549 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78460f04-e538-413b-89fb-3cf5cc7c0286 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:51.944849 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78460f04-e538-413b-89fb-3cf5cc7c0286 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:51.945152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78460f04-e538-413b-89fb-3cf5cc7c0286 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:51.957407 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78460f04-e538-413b-89fb-3cf5cc7c0286 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:51.959598 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78460f04-e538-413b-89fb-3cf5cc7c0286 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:51.972621 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1158/2313] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:11:52.295606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d0d36-e6a8-43c0-9209-1959920b41de None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:52.297290 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bb8d0d36-e6a8-43c0-9209-1959920b41de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:52.310362 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d0d36-e6a8-43c0-9209-1959920b41de None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:52.316626 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2b799-a15b-47db-9c96-b8dd02d88a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:52.317643 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2b799-a15b-47db-9c96-b8dd02d88a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:52.317931 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2b799-a15b-47db-9c96-b8dd02d88a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:52.318210 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2b799-a15b-47db-9c96-b8dd02d88a0e admin admin] PATH_INFO: `/v3/projects/8b5e20dc231945c89a98821684f9ed2f` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:52.319489 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2b799-a15b-47db-9c96-b8dd02d88a0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b5e20dc231945c89a98821684f9ed2f)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:52.321579 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2b799-a15b-47db-9c96-b8dd02d88a0e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:52.380988 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8ef2b799-a15b-47db-9c96-b8dd02d88a0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b5e20dc231945c89a98821684f9ed2f', 'request_id': 'req-8ef2b799-a15b-47db-9c96-b8dd02d88a0e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:52.388456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8ef2b799-a15b-47db-9c96-b8dd02d88a0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b5e20dc231945c89a98821684f9ed2f', 'request_id': 'req-8ef2b799-a15b-47db-9c96-b8dd02d88a0e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:52.397012 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8ef2b799-a15b-47db-9c96-b8dd02d88a0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b5e20dc231945c89a98821684f9ed2f', 'request_id': 'req-8ef2b799-a15b-47db-9c96-b8dd02d88a0e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:52.399658 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:52.411123 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8ef2b799-a15b-47db-9c96-b8dd02d88a0e admin admin] The token cache is being invalidate because project 8b5e20dc231945c89a98821684f9ed2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:11:52.412005 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8ef2b799-a15b-47db-9c96-b8dd02d88a0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:52.413106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8ef2b799-a15b-47db-9c96-b8dd02d88a0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:52.415209 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1156/2314] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:11:52 2022] DELETE /identity/v3/projects/8b5e20dc231945c89a98821684f9ed2f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:52.419231 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78460f04-e538-413b-89fb-3cf5cc7c0286 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:52.429618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78460f04-e538-413b-89fb-3cf5cc7c0286 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:52.507422 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f35c1b-9004-4d2c-ae5e-4f03a8a4e17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:52.508599 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f35c1b-9004-4d2c-ae5e-4f03a8a4e17e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:52.508965 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f35c1b-9004-4d2c-ae5e-4f03a8a4e17e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:52.509317 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f35c1b-9004-4d2c-ae5e-4f03a8a4e17e admin admin] PATH_INFO: `/v3/users/eb06a32d7fa44cbe83b9ac83ef9c40c0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:52.510691 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f35c1b-9004-4d2c-ae5e-4f03a8a4e17e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb06a32d7fa44cbe83b9ac83ef9c40c0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:52.513807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f35c1b-9004-4d2c-ae5e-4f03a8a4e17e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:52.605739 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d5f35c1b-9004-4d2c-ae5e-4f03a8a4e17e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb06a32d7fa44cbe83b9ac83ef9c40c0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:52.617476 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d5f35c1b-9004-4d2c-ae5e-4f03a8a4e17e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb06a32d7fa44cbe83b9ac83ef9c40c0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:52.626773 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d5f35c1b-9004-4d2c-ae5e-4f03a8a4e17e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb06a32d7fa44cbe83b9ac83ef9c40c0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:52.628282 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d5f35c1b-9004-4d2c-ae5e-4f03a8a4e17e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb06a32d7fa44cbe83b9ac83ef9c40c0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:52.646623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d5f35c1b-9004-4d2c-ae5e-4f03a8a4e17e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb06a32d7fa44cbe83b9ac83ef9c40c0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:52.653792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1159/2315] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:11:52 2022] DELETE /identity/v3/users/eb06a32d7fa44cbe83b9ac83ef9c40c0 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:53.018558 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2b799-a15b-47db-9c96-b8dd02d88a0e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:53.020012 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8ef2b799-a15b-47db-9c96-b8dd02d88a0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:53.047909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2b799-a15b-47db-9c96-b8dd02d88a0e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:53.140328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f6318-bf9e-4fc1-9080-25e9da6b5f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:53.141210 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f6318-bf9e-4fc1-9080-25e9da6b5f2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:53.141500 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f6318-bf9e-4fc1-9080-25e9da6b5f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:53.141772 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f6318-bf9e-4fc1-9080-25e9da6b5f2e admin admin] PATH_INFO: `/v3/projects/5231d296ede54f66ad6122fcf729b6b3` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:53.143763 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f6318-bf9e-4fc1-9080-25e9da6b5f2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5231d296ede54f66ad6122fcf729b6b3)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:53.146331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f6318-bf9e-4fc1-9080-25e9da6b5f2e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:53.215797 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5d3f6318-bf9e-4fc1-9080-25e9da6b5f2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5231d296ede54f66ad6122fcf729b6b3', 'request_id': 'req-5d3f6318-bf9e-4fc1-9080-25e9da6b5f2e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:53.224745 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5d3f6318-bf9e-4fc1-9080-25e9da6b5f2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5231d296ede54f66ad6122fcf729b6b3', 'request_id': 'req-5d3f6318-bf9e-4fc1-9080-25e9da6b5f2e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:53.236055 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5d3f6318-bf9e-4fc1-9080-25e9da6b5f2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5231d296ede54f66ad6122fcf729b6b3', 'request_id': 'req-5d3f6318-bf9e-4fc1-9080-25e9da6b5f2e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:53.253705 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5d3f6318-bf9e-4fc1-9080-25e9da6b5f2e admin admin] The token cache is being invalidate because project 5231d296ede54f66ad6122fcf729b6b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:11:53.254044 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5d3f6318-bf9e-4fc1-9080-25e9da6b5f2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:53.254705 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5d3f6318-bf9e-4fc1-9080-25e9da6b5f2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:53.258180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1157/2316] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:11:53 2022] DELETE /identity/v3/projects/5231d296ede54f66ad6122fcf729b6b3 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:53.265175 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61f992-8451-4579-b708-a92b92db55a5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:53.265175 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61f992-8451-4579-b708-a92b92db55a5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:53.265175 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61f992-8451-4579-b708-a92b92db55a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:53.286981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fa61f992-8451-4579-b708-a92b92db55a5 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:53.288267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fa61f992-8451-4579-b708-a92b92db55a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:53.368071 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1160/2317] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:53.376740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac841d15-7819-45e9-bd58-862890b172cc None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:53.377777 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac841d15-7819-45e9-bd58-862890b172cc None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:53.378272 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac841d15-7819-45e9-bd58-862890b172cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:53.401188 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ac841d15-7819-45e9-bd58-862890b172cc None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:53.402911 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ac841d15-7819-45e9-bd58-862890b172cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:53.416329 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1158/2318] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:11:53.421231 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61f992-8451-4579-b708-a92b92db55a5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:53.434544 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61f992-8451-4579-b708-a92b92db55a5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:53.441038 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c9a0d-e50f-471d-b67b-5ce9c3bc92e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:53.442019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c9a0d-e50f-471d-b67b-5ce9c3bc92e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:53.442804 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c9a0d-e50f-471d-b67b-5ce9c3bc92e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:53.443315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c9a0d-e50f-471d-b67b-5ce9c3bc92e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:53.444318 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c9a0d-e50f-471d-b67b-5ce9c3bc92e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:11:53.444864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c9a0d-e50f-471d-b67b-5ce9c3bc92e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:53.446596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c9a0d-e50f-471d-b67b-5ce9c3bc92e0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:53.449535 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:53.468476 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac841d15-7819-45e9-bd58-862890b172cc None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:53.478934 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1161/2319] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:11:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:11:53.479132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac841d15-7819-45e9-bd58-862890b172cc None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:53.484627 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f7bef-e079-492b-b0a8-d83f3bff157d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:53.485547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f7bef-e079-492b-b0a8-d83f3bff157d admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:53.485837 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f7bef-e079-492b-b0a8-d83f3bff157d admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:53.486114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f7bef-e079-492b-b0a8-d83f3bff157d admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:53.487326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f7bef-e079-492b-b0a8-d83f3bff157d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:53.489420 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f7bef-e079-492b-b0a8-d83f3bff157d admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:53.539985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1159/2320] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:11:53 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:11:53.542383 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c9a0d-e50f-471d-b67b-5ce9c3bc92e0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:53.561059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c9a0d-e50f-471d-b67b-5ce9c3bc92e0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:53.660898 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1be0b7-f130-4c90-978c-7ae451e52f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:53.661896 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1be0b7-f130-4c90-978c-7ae451e52f6d admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:53.662347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1be0b7-f130-4c90-978c-7ae451e52f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:53.662776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1be0b7-f130-4c90-978c-7ae451e52f6d admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:53.664544 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1be0b7-f130-4c90-978c-7ae451e52f6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:53.670916 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1be0b7-f130-4c90-978c-7ae451e52f6d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:53.718888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1162/2321] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:11:53 2022] POST /identity/v3/users => generated 287 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:11:53.721169 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f7bef-e079-492b-b0a8-d83f3bff157d admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:53.735508 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f7bef-e079-492b-b0a8-d83f3bff157d admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:53.745835 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c6ac0a-218e-4c77-b975-1fb7981bed53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:53.747254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6ac0a-218e-4c77-b975-1fb7981bed53 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:53.747763 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6ac0a-218e-4c77-b975-1fb7981bed53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:53.748308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6ac0a-218e-4c77-b975-1fb7981bed53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:53.749305 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c6ac0a-218e-4c77-b975-1fb7981bed53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:53.751306 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c6ac0a-218e-4c77-b975-1fb7981bed53 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:53.805052 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1160/2322] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:11:53.805952 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1be0b7-f130-4c90-978c-7ae451e52f6d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:53.818856 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1be0b7-f130-4c90-978c-7ae451e52f6d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:53.858555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c00b65-b56b-481f-8844-0ea154fa66da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:53.859274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c00b65-b56b-481f-8844-0ea154fa66da admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:53.859517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c00b65-b56b-481f-8844-0ea154fa66da admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:53.859790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c00b65-b56b-481f-8844-0ea154fa66da admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:53.860750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c00b65-b56b-481f-8844-0ea154fa66da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:53.863172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c00b65-b56b-481f-8844-0ea154fa66da admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:53.886436 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1163/2323] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:11:53.890599 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c6ac0a-218e-4c77-b975-1fb7981bed53 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:53.903337 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c6ac0a-218e-4c77-b975-1fb7981bed53 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:53.925452 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1b9fb-76e0-4139-98f4-e6054755473d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:53.926571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1b9fb-76e0-4139-98f4-e6054755473d admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:53.926863 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1b9fb-76e0-4139-98f4-e6054755473d admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:53.927147 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1b9fb-76e0-4139-98f4-e6054755473d admin admin] PATH_INFO: `/v3/projects/be0b01666ba44f189a363c00ba997d71/users/7e8f2820e15141dfb31bbeb1d2e0a23f/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:53.965471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1b9fb-76e0-4139-98f4-e6054755473d admin admin] RBAC: Authorizing `identity:create_grant(project_id=be0b01666ba44f189a363c00ba997d71, user_id=7e8f2820e15141dfb31bbeb1d2e0a23f, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:53.969519 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1b9fb-76e0-4139-98f4-e6054755473d admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:53.992614 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1161/2324] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:11:53 2022] PUT /identity/v3/projects/be0b01666ba44f189a363c00ba997d71/users/7e8f2820e15141dfb31bbeb1d2e0a23f/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:11:54.003212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9dd3c-b4c8-4ee7-b04b-ce2c1a2ca0eb None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:54.003910 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9dd3c-b4c8-4ee7-b04b-ce2c1a2ca0eb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:54.004575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9dd3c-b4c8-4ee7-b04b-ce2c1a2ca0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:54.073472 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-55b9dd3c-b4c8-4ee7-b04b-ce2c1a2ca0eb None None] MFA Rules not processed for user `7e8f2820e15141dfb31bbeb1d2e0a23f`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:54.075549 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-55b9dd3c-b4c8-4ee7-b04b-ce2c1a2ca0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:54.223461 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1164/2325] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:54.235341 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1b9fb-76e0-4139-98f4-e6054755473d admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:54.237216 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f8d1b9fb-76e0-4139-98f4-e6054755473d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:54.276963 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:54.326809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1b9fb-76e0-4139-98f4-e6054755473d admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:54.334117 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7266c1-565f-4b64-806b-554efe925581 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:54.335541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7266c1-565f-4b64-806b-554efe925581 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:54.335836 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7266c1-565f-4b64-806b-554efe925581 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:54.336236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7266c1-565f-4b64-806b-554efe925581 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:54.347716 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7266c1-565f-4b64-806b-554efe925581 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:54.349384 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7266c1-565f-4b64-806b-554efe925581 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:54.360985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1162/2326] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:54.391875 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b9dd3c-b4c8-4ee7-b04b-ce2c1a2ca0eb None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:54.487206 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b9dd3c-b4c8-4ee7-b04b-ce2c1a2ca0eb None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:54.498281 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00535706-178e-4f8f-bf14-c5d034d06cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:54.506806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00535706-178e-4f8f-bf14-c5d034d06cd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:54.507422 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00535706-178e-4f8f-bf14-c5d034d06cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:54.507753 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00535706-178e-4f8f-bf14-c5d034d06cd4 admin admin] PATH_INFO: `/v3/users/7e8f2820e15141dfb31bbeb1d2e0a23f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:54.525264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00535706-178e-4f8f-bf14-c5d034d06cd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e8f2820e15141dfb31bbeb1d2e0a23f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:54.529276 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00535706-178e-4f8f-bf14-c5d034d06cd4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:54.539489 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:54.620804 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-00535706-178e-4f8f-bf14-c5d034d06cd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e8f2820e15141dfb31bbeb1d2e0a23f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:54.634379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-00535706-178e-4f8f-bf14-c5d034d06cd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e8f2820e15141dfb31bbeb1d2e0a23f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:54.644639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-00535706-178e-4f8f-bf14-c5d034d06cd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e8f2820e15141dfb31bbeb1d2e0a23f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:54.645329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-00535706-178e-4f8f-bf14-c5d034d06cd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e8f2820e15141dfb31bbeb1d2e0a23f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:54.659391 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-00535706-178e-4f8f-bf14-c5d034d06cd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e8f2820e15141dfb31bbeb1d2e0a23f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:54.666367 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1165/2327] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:11:54 2022] DELETE /identity/v3/users/7e8f2820e15141dfb31bbeb1d2e0a23f => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:54.671491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c281219-11a3-4ceb-af26-ed8b4a3503ce None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:54.672031 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c281219-11a3-4ceb-af26-ed8b4a3503ce None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:54.672031 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c281219-11a3-4ceb-af26-ed8b4a3503ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:54.696797 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-0c281219-11a3-4ceb-af26-ed8b4a3503ce None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:54.698007 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0c281219-11a3-4ceb-af26-ed8b4a3503ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:54.780618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1163/2328] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:54.791750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00535706-178e-4f8f-bf14-c5d034d06cd4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:54.793329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-00535706-178e-4f8f-bf14-c5d034d06cd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:54.855189 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00535706-178e-4f8f-bf14-c5d034d06cd4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:54.868643 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314cd81a-7ee1-4cf9-9232-fc08462c6adb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:54.869485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cd81a-7ee1-4cf9-9232-fc08462c6adb service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:54.869611 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cd81a-7ee1-4cf9-9232-fc08462c6adb service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:54.869872 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cd81a-7ee1-4cf9-9232-fc08462c6adb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:54.881328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314cd81a-7ee1-4cf9-9232-fc08462c6adb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:54.882518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314cd81a-7ee1-4cf9-9232-fc08462c6adb service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:54.908687 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1166/2329] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:11:55.540742 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c281219-11a3-4ceb-af26-ed8b4a3503ce None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:55.543231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0c281219-11a3-4ceb-af26-ed8b4a3503ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:55.577602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c281219-11a3-4ceb-af26-ed8b4a3503ce None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:55.596502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e97ea4-7e20-465c-bcce-a40a045f2132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:55.598546 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e97ea4-7e20-465c-bcce-a40a045f2132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:55.598837 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e97ea4-7e20-465c-bcce-a40a045f2132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:55.599144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e97ea4-7e20-465c-bcce-a40a045f2132 admin admin] PATH_INFO: `/v3/projects/be0b01666ba44f189a363c00ba997d71` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:55.601159 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e97ea4-7e20-465c-bcce-a40a045f2132 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be0b01666ba44f189a363c00ba997d71)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:55.603141 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e97ea4-7e20-465c-bcce-a40a045f2132 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:55.674445 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d4e97ea4-7e20-465c-bcce-a40a045f2132 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be0b01666ba44f189a363c00ba997d71', 'request_id': 'req-d4e97ea4-7e20-465c-bcce-a40a045f2132'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:55.681610 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d4e97ea4-7e20-465c-bcce-a40a045f2132 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be0b01666ba44f189a363c00ba997d71', 'request_id': 'req-d4e97ea4-7e20-465c-bcce-a40a045f2132'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:55.706658 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d4e97ea4-7e20-465c-bcce-a40a045f2132 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be0b01666ba44f189a363c00ba997d71', 'request_id': 'req-d4e97ea4-7e20-465c-bcce-a40a045f2132'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:55.725859 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d4e97ea4-7e20-465c-bcce-a40a045f2132 admin admin] The token cache is being invalidate because project be0b01666ba44f189a363c00ba997d71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:11:55.726144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d4e97ea4-7e20-465c-bcce-a40a045f2132 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:55.727781 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d4e97ea4-7e20-465c-bcce-a40a045f2132 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:11:55.730964 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1164/2330] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:11:55 2022] DELETE /identity/v3/projects/be0b01666ba44f189a363c00ba997d71 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:11:55.735901 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a333b2-8977-4f76-a271-80679fc31396 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:55.736456 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a333b2-8977-4f76-a271-80679fc31396 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:55.736665 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a333b2-8977-4f76-a271-80679fc31396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:55.764989 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-70a333b2-8977-4f76-a271-80679fc31396 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:55.766326 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-70a333b2-8977-4f76-a271-80679fc31396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:55.894973 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1167/2331] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:55.897113 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83669b20-f765-4011-bc41-b0a3d4b05dbc None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:55.897113 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83669b20-f765-4011-bc41-b0a3d4b05dbc None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:55.897113 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83669b20-f765-4011-bc41-b0a3d4b05dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:55.936338 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-83669b20-f765-4011-bc41-b0a3d4b05dbc None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:55.936338 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-83669b20-f765-4011-bc41-b0a3d4b05dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:55.970303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1165/2332] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:11:55.975896 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a333b2-8977-4f76-a271-80679fc31396 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:55.997673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a333b2-8977-4f76-a271-80679fc31396 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:56.006403 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e03db-8d46-4688-876d-fadca51d2fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:56.007241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e03db-8d46-4688-876d-fadca51d2fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:56.007529 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e03db-8d46-4688-876d-fadca51d2fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:56.007808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e03db-8d46-4688-876d-fadca51d2fb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:56.008601 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150e03db-8d46-4688-876d-fadca51d2fb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:11:56.008912 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150e03db-8d46-4688-876d-fadca51d2fb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:56.010558 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150e03db-8d46-4688-876d-fadca51d2fb7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:56.027706 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1168/2333] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:11:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:11:56.032777 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83669b20-f765-4011-bc41-b0a3d4b05dbc None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:56.050083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83669b20-f765-4011-bc41-b0a3d4b05dbc None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:56.056592 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cece8-371c-4db4-8d4a-52f3b5337301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:56.057280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cece8-371c-4db4-8d4a-52f3b5337301 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:56.057574 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cece8-371c-4db4-8d4a-52f3b5337301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:56.057858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cece8-371c-4db4-8d4a-52f3b5337301 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:56.058852 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466cece8-371c-4db4-8d4a-52f3b5337301 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:56.062135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466cece8-371c-4db4-8d4a-52f3b5337301 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:56.158251 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1166/2334] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:11:56 2022] POST /identity/v3/projects => generated 389 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:11:56.159806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e03db-8d46-4688-876d-fadca51d2fb7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:56.199564 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e03db-8d46-4688-876d-fadca51d2fb7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:56.220539 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:56.288003 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026823a-ea7e-464e-aa0a-8baef05025ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:56.289038 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026823a-ea7e-464e-aa0a-8baef05025ce admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:56.289919 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026823a-ea7e-464e-aa0a-8baef05025ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:56.290269 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026823a-ea7e-464e-aa0a-8baef05025ce admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:56.291084 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026823a-ea7e-464e-aa0a-8baef05025ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:56.294334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026823a-ea7e-464e-aa0a-8baef05025ce admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:56.326458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1169/2335] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:11:56 2022] POST /identity/v3/users => generated 295 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:11:56.330691 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cece8-371c-4db4-8d4a-52f3b5337301 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:56.342657 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cece8-371c-4db4-8d4a-52f3b5337301 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:56.349142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84f742-3aa8-4f24-8b2c-69922be4c1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:56.349142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84f742-3aa8-4f24-8b2c-69922be4c1e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:56.349142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84f742-3aa8-4f24-8b2c-69922be4c1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:56.349433 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84f742-3aa8-4f24-8b2c-69922be4c1e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:56.350132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd84f742-3aa8-4f24-8b2c-69922be4c1e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:56.351985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd84f742-3aa8-4f24-8b2c-69922be4c1e0 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:56.363435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1167/2336] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:11:56.367295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026823a-ea7e-464e-aa0a-8baef05025ce admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:56.380695 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026823a-ea7e-464e-aa0a-8baef05025ce admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:56.392067 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a9950e-f2dc-4bd4-88c9-9770fdf9cbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:56.393333 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9950e-f2dc-4bd4-88c9-9770fdf9cbbb admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:56.393628 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9950e-f2dc-4bd4-88c9-9770fdf9cbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:56.394268 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a9950e-f2dc-4bd4-88c9-9770fdf9cbbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:56.394656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a9950e-f2dc-4bd4-88c9-9770fdf9cbbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:56.396590 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a9950e-f2dc-4bd4-88c9-9770fdf9cbbb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:56.418528 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1170/2337] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:11:56.422577 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84f742-3aa8-4f24-8b2c-69922be4c1e0 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:56.437855 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84f742-3aa8-4f24-8b2c-69922be4c1e0 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:56.444785 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b75d57-2b6b-4701-a192-9ebb9b5fb728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:56.445683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b75d57-2b6b-4701-a192-9ebb9b5fb728 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:56.445965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b75d57-2b6b-4701-a192-9ebb9b5fb728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:56.446245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b75d57-2b6b-4701-a192-9ebb9b5fb728 admin admin] PATH_INFO: `/v3/projects/bc40e69ef09e4017b43d9eb687c298a9/users/a33dd0989b664e9aa9946e397fccbfb1/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:56.451791 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:56.469025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b75d57-2b6b-4701-a192-9ebb9b5fb728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc40e69ef09e4017b43d9eb687c298a9, user_id=a33dd0989b664e9aa9946e397fccbfb1, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:56.476972 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b75d57-2b6b-4701-a192-9ebb9b5fb728 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:56.505201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1168/2338] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:11:56 2022] PUT /identity/v3/projects/bc40e69ef09e4017b43d9eb687c298a9/users/a33dd0989b664e9aa9946e397fccbfb1/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:11:56.516343 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4ded2-3df9-4db7-b238-7124d1e49718 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:56.516649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4ded2-3df9-4db7-b238-7124d1e49718 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:56.516939 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4ded2-3df9-4db7-b238-7124d1e49718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:56.552608 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8ec4ded2-3df9-4db7-b238-7124d1e49718 None None] MFA Rules not processed for user `a33dd0989b664e9aa9946e397fccbfb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:56.553984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8ec4ded2-3df9-4db7-b238-7124d1e49718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:56.671462 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1171/2339] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:11:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:56.682583 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b75d57-2b6b-4701-a192-9ebb9b5fb728 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:56.687751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-27b75d57-2b6b-4701-a192-9ebb9b5fb728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:56.778868 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b75d57-2b6b-4701-a192-9ebb9b5fb728 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:56.783454 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4bbf6-7177-4986-9e81-32297bbecf38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:56.786234 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4bbf6-7177-4986-9e81-32297bbecf38 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:56.786861 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4bbf6-7177-4986-9e81-32297bbecf38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:56.786861 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4bbf6-7177-4986-9e81-32297bbecf38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:56.798405 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b4bbf6-7177-4986-9e81-32297bbecf38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:56.800546 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b4bbf6-7177-4986-9e81-32297bbecf38 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:56.819303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1169/2340] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:12:06.667067 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=60,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:06.806237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=32,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:07.860865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256ba74-2bfc-497d-aa5f-4f6dc611ee21 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:07.861319 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256ba74-2bfc-497d-aa5f-4f6dc611ee21 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:07.861419 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256ba74-2bfc-497d-aa5f-4f6dc611ee21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:07.879388 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6256ba74-2bfc-497d-aa5f-4f6dc611ee21 None None] MFA Rules not processed for user `94d824bffc2c4af299c537056bc21734`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:07.880607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6256ba74-2bfc-497d-aa5f-4f6dc611ee21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:07.935877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1172/2341] 10.222.0.94 () {58 vars in 967 bytes} [Mon Jul 18 14:12:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:07.951621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4bbf6-7177-4986-9e81-32297bbecf38 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:07.953390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a0b4bbf6-7177-4986-9e81-32297bbecf38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:08.028094 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4bbf6-7177-4986-9e81-32297bbecf38 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:08.033602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d6e3a-c0a5-40f8-917c-4ee70b885c41 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbf513ee25374e49b3d8ce28843e393a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:08.034266 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d6e3a-c0a5-40f8-917c-4ee70b885c41 service swift] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:08.034548 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d6e3a-c0a5-40f8-917c-4ee70b885c41 service swift] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:08.034829 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d6e3a-c0a5-40f8-917c-4ee70b885c41 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:08.044031 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d6e3a-c0a5-40f8-917c-4ee70b885c41 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:08.045655 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d6e3a-c0a5-40f8-917c-4ee70b885c41 service swift] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:08.055073 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1170/2342] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Jul 18 14:12:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:12:17.928484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=25 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:18.051363 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=32 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:19.777189 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9bee5-eceb-4508-8656-dbd0de5538f0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:19.777716 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9bee5-eceb-4508-8656-dbd0de5538f0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:19.777818 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9bee5-eceb-4508-8656-dbd0de5538f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:19.795627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-edf9bee5-eceb-4508-8656-dbd0de5538f0 None None] MFA Rules not processed for user `da8ecff90e714e4e961f8a8104143f62`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:19.796878 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-edf9bee5-eceb-4508-8656-dbd0de5538f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:19.854580 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1173/2343] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:12:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:25.823541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d6e3a-c0a5-40f8-917c-4ee70b885c41 service swift] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:25.825640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-507d6e3a-c0a5-40f8-917c-4ee70b885c41 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:25.907080 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d6e3a-c0a5-40f8-917c-4ee70b885c41 service swift] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:25.912908 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718137e-61e7-4de0-a05e-124cc1c3f9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:25.913799 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718137e-61e7-4de0-a05e-124cc1c3f9a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:25.914088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718137e-61e7-4de0-a05e-124cc1c3f9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:25.914366 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718137e-61e7-4de0-a05e-124cc1c3f9a3 admin admin] PATH_INFO: `/v3/users/936b6d72268244d09c079532a24a969c` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:25.915569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718137e-61e7-4de0-a05e-124cc1c3f9a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=936b6d72268244d09c079532a24a969c)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:25.917958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718137e-61e7-4de0-a05e-124cc1c3f9a3 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:25.974686 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f718137e-61e7-4de0-a05e-124cc1c3f9a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '936b6d72268244d09c079532a24a969c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:25.982446 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f718137e-61e7-4de0-a05e-124cc1c3f9a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '936b6d72268244d09c079532a24a969c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:25.991754 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f718137e-61e7-4de0-a05e-124cc1c3f9a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '936b6d72268244d09c079532a24a969c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:25.992536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f718137e-61e7-4de0-a05e-124cc1c3f9a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '936b6d72268244d09c079532a24a969c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:26.006128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f718137e-61e7-4de0-a05e-124cc1c3f9a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '936b6d72268244d09c079532a24a969c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:26.012478 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1171/2344] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:12:25 2022] DELETE /identity/v3/users/936b6d72268244d09c079532a24a969c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:12:26.192231 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9bee5-eceb-4508-8656-dbd0de5538f0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:26.203871 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9bee5-eceb-4508-8656-dbd0de5538f0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:26.270049 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41aac4-3cd3-4a89-a7c1-4b13085438cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:26.270970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41aac4-3cd3-4a89-a7c1-4b13085438cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:26.271792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41aac4-3cd3-4a89-a7c1-4b13085438cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:26.271792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41aac4-3cd3-4a89-a7c1-4b13085438cf admin admin] PATH_INFO: `/v3/projects/2e0380aba6504b61af9e82a6a72d0211` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:26.272816 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41aac4-3cd3-4a89-a7c1-4b13085438cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e0380aba6504b61af9e82a6a72d0211)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:26.274750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41aac4-3cd3-4a89-a7c1-4b13085438cf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:26.328378 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cf41aac4-3cd3-4a89-a7c1-4b13085438cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e0380aba6504b61af9e82a6a72d0211', 'request_id': 'req-cf41aac4-3cd3-4a89-a7c1-4b13085438cf'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:26.337070 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cf41aac4-3cd3-4a89-a7c1-4b13085438cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e0380aba6504b61af9e82a6a72d0211', 'request_id': 'req-cf41aac4-3cd3-4a89-a7c1-4b13085438cf'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:26.348028 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cf41aac4-3cd3-4a89-a7c1-4b13085438cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e0380aba6504b61af9e82a6a72d0211', 'request_id': 'req-cf41aac4-3cd3-4a89-a7c1-4b13085438cf'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:26.360697 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cf41aac4-3cd3-4a89-a7c1-4b13085438cf admin admin] The token cache is being invalidate because project 2e0380aba6504b61af9e82a6a72d0211 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:12:26.360964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cf41aac4-3cd3-4a89-a7c1-4b13085438cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:26.361638 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cf41aac4-3cd3-4a89-a7c1-4b13085438cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:26.363663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1174/2345] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:12:26 2022] DELETE /identity/v3/projects/2e0380aba6504b61af9e82a6a72d0211 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:12:26.368488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee1d21-0bdc-4ee1-9e17-8ec6ddd42a0a None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:26.368781 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee1d21-0bdc-4ee1-9e17-8ec6ddd42a0a None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:26.369063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee1d21-0bdc-4ee1-9e17-8ec6ddd42a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:26.387717 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ccee1d21-0bdc-4ee1-9e17-8ec6ddd42a0a None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:26.390923 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ccee1d21-0bdc-4ee1-9e17-8ec6ddd42a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:26.470248 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1172/2346] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:12:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:26.473420 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16808a-8931-468f-9081-51fc3868eb2b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:26.474153 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16808a-8931-468f-9081-51fc3868eb2b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:26.474454 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16808a-8931-468f-9081-51fc3868eb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:26.498281 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0d16808a-8931-468f-9081-51fc3868eb2b None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:26.499679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0d16808a-8931-468f-9081-51fc3868eb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:26.513713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee1d21-0bdc-4ee1-9e17-8ec6ddd42a0a None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:26.516024 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1175/2347] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:12:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:26.526651 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee1d21-0bdc-4ee1-9e17-8ec6ddd42a0a None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:26.533100 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d38b79-98a5-4f1f-9892-e86ad959f960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:26.534180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d38b79-98a5-4f1f-9892-e86ad959f960 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:26.534547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d38b79-98a5-4f1f-9892-e86ad959f960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:26.534928 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d38b79-98a5-4f1f-9892-e86ad959f960 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:26.536016 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d38b79-98a5-4f1f-9892-e86ad959f960 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:12:26.536365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d38b79-98a5-4f1f-9892-e86ad959f960 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:26.538291 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d38b79-98a5-4f1f-9892-e86ad959f960 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:26.554536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1173/2348] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:12:26.555710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16808a-8931-468f-9081-51fc3868eb2b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:26.568581 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16808a-8931-468f-9081-51fc3868eb2b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:26.574400 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaca625-df6c-4905-abb8-376522b7a4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:26.575086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaca625-df6c-4905-abb8-376522b7a4c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:26.575482 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaca625-df6c-4905-abb8-376522b7a4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:26.575780 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaca625-df6c-4905-abb8-376522b7a4c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:26.576884 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaca625-df6c-4905-abb8-376522b7a4c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:26.579270 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaca625-df6c-4905-abb8-376522b7a4c0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:26.612516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1176/2349] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:12:26 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:12:26.616486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d38b79-98a5-4f1f-9892-e86ad959f960 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:26.626504 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d38b79-98a5-4f1f-9892-e86ad959f960 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:26.630313 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:26.686722 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5623fe-dc84-4a0b-abaa-85596be816a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:26.688491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5623fe-dc84-4a0b-abaa-85596be816a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:26.688830 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5623fe-dc84-4a0b-abaa-85596be816a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:26.689116 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5623fe-dc84-4a0b-abaa-85596be816a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:26.689825 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5623fe-dc84-4a0b-abaa-85596be816a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:26.691798 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5623fe-dc84-4a0b-abaa-85596be816a9 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:26.715620 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1174/2350] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:12:26 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:12:26.716938 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaca625-df6c-4905-abb8-376522b7a4c0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:26.728149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaca625-df6c-4905-abb8-376522b7a4c0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:26.734213 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1dbb8-bfe6-446b-b22b-6882a17e14f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:26.734932 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1dbb8-bfe6-446b-b22b-6882a17e14f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:26.735223 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1dbb8-bfe6-446b-b22b-6882a17e14f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:26.735503 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1dbb8-bfe6-446b-b22b-6882a17e14f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:26.736264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1dbb8-bfe6-446b-b22b-6882a17e14f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:26.738022 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1dbb8-bfe6-446b-b22b-6882a17e14f6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:26.739507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:26.753265 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1177/2351] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:12:26.756321 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5623fe-dc84-4a0b-abaa-85596be816a9 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:26.767676 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5623fe-dc84-4a0b-abaa-85596be816a9 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:26.774399 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f2176-ba11-4f64-8fa9-704a08dd1514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:26.775786 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f2176-ba11-4f64-8fa9-704a08dd1514 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:26.776165 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f2176-ba11-4f64-8fa9-704a08dd1514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:26.776630 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f2176-ba11-4f64-8fa9-704a08dd1514 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:26.777441 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f2176-ba11-4f64-8fa9-704a08dd1514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:26.779252 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f2176-ba11-4f64-8fa9-704a08dd1514 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:26.798164 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1175/2352] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:12:26.799796 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1dbb8-bfe6-446b-b22b-6882a17e14f6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:26.815086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1dbb8-bfe6-446b-b22b-6882a17e14f6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:26.819919 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c77c94-7b06-4ac3-803a-23b7a40441cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:26.820946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c77c94-7b06-4ac3-803a-23b7a40441cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:26.821261 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c77c94-7b06-4ac3-803a-23b7a40441cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:26.821554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c77c94-7b06-4ac3-803a-23b7a40441cc admin admin] PATH_INFO: `/v3/projects/cb3c985bd0514ed098260ca5b836ae66/users/67814b29011342ba90e603a1d389bea4/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:26.837058 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c77c94-7b06-4ac3-803a-23b7a40441cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb3c985bd0514ed098260ca5b836ae66, user_id=67814b29011342ba90e603a1d389bea4, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:26.839749 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c77c94-7b06-4ac3-803a-23b7a40441cc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:26.853478 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1178/2353] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:12:26 2022] PUT /identity/v3/projects/cb3c985bd0514ed098260ca5b836ae66/users/67814b29011342ba90e603a1d389bea4/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:12:26.864071 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebcd9a0-558d-44ce-bee8-734e75669549 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:26.864540 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebcd9a0-558d-44ce-bee8-734e75669549 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:26.864699 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebcd9a0-558d-44ce-bee8-734e75669549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:26.893192 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-bebcd9a0-558d-44ce-bee8-734e75669549 None None] MFA Rules not processed for user `67814b29011342ba90e603a1d389bea4`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:26.893594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bebcd9a0-558d-44ce-bee8-734e75669549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:26.973132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1176/2354] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:12:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:26.985965 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c77c94-7b06-4ac3-803a-23b7a40441cc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:26.989161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-56c77c94-7b06-4ac3-803a-23b7a40441cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:27.059657 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c77c94-7b06-4ac3-803a-23b7a40441cc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:27.063934 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a8b8e-ede3-4151-a883-c1a9dd6481c0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '503a8228d4754c309fc3f365f4cfc61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:27.064628 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a8b8e-ede3-4151-a883-c1a9dd6481c0 service glance] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:27.064919 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a8b8e-ede3-4151-a883-c1a9dd6481c0 service glance] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:27.065199 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a8b8e-ede3-4151-a883-c1a9dd6481c0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:27.074795 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a8b8e-ede3-4151-a883-c1a9dd6481c0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:27.076327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a8b8e-ede3-4151-a883-c1a9dd6481c0 service glance] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:27.087620 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1179/2355] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:12:26 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:12:29.831738 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebcd9a0-558d-44ce-bee8-734e75669549 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:29.833729 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bebcd9a0-558d-44ce-bee8-734e75669549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:29.898385 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebcd9a0-558d-44ce-bee8-734e75669549 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:29.902923 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03aa9d-eec0-4d24-af78-9ca8d1977418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:29.903790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03aa9d-eec0-4d24-af78-9ca8d1977418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:29.904107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03aa9d-eec0-4d24-af78-9ca8d1977418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:29.904385 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03aa9d-eec0-4d24-af78-9ca8d1977418 admin admin] PATH_INFO: `/v3/users/a33dd0989b664e9aa9946e397fccbfb1` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:29.905556 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03aa9d-eec0-4d24-af78-9ca8d1977418 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a33dd0989b664e9aa9946e397fccbfb1)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:29.908104 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03aa9d-eec0-4d24-af78-9ca8d1977418 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:29.925298 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:29.958528 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4f03aa9d-eec0-4d24-af78-9ca8d1977418 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a33dd0989b664e9aa9946e397fccbfb1'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:29.965701 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4f03aa9d-eec0-4d24-af78-9ca8d1977418 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a33dd0989b664e9aa9946e397fccbfb1'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:29.973288 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4f03aa9d-eec0-4d24-af78-9ca8d1977418 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a33dd0989b664e9aa9946e397fccbfb1'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:29.973931 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4f03aa9d-eec0-4d24-af78-9ca8d1977418 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a33dd0989b664e9aa9946e397fccbfb1'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:29.985800 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4f03aa9d-eec0-4d24-af78-9ca8d1977418 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a33dd0989b664e9aa9946e397fccbfb1'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:29.991681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1177/2356] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:12:29 2022] DELETE /identity/v3/users/a33dd0989b664e9aa9946e397fccbfb1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:12:29.996515 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d41b6-f868-4e1e-9ab2-4fd598f360dd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:29.997089 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d41b6-f868-4e1e-9ab2-4fd598f360dd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:29.997089 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d41b6-f868-4e1e-9ab2-4fd598f360dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:30.014025 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4b7d41b6-f868-4e1e-9ab2-4fd598f360dd None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:30.015230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4b7d41b6-f868-4e1e-9ab2-4fd598f360dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:30.075952 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1180/2357] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:12:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:30.086108 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03aa9d-eec0-4d24-af78-9ca8d1977418 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:30.087530 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4f03aa9d-eec0-4d24-af78-9ca8d1977418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:30.145422 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03aa9d-eec0-4d24-af78-9ca8d1977418 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:30.149138 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2683f291-2641-42bc-a4e3-9bf6d2413b58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:30.149729 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683f291-2641-42bc-a4e3-9bf6d2413b58 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:30.150020 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683f291-2641-42bc-a4e3-9bf6d2413b58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:30.150302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683f291-2641-42bc-a4e3-9bf6d2413b58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:30.159379 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2683f291-2641-42bc-a4e3-9bf6d2413b58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:30.160908 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2683f291-2641-42bc-a4e3-9bf6d2413b58 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:30.171219 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1178/2358] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:12:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:12:30.478945 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d41b6-f868-4e1e-9ab2-4fd598f360dd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:30.480522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4b7d41b6-f868-4e1e-9ab2-4fd598f360dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:30.494380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d41b6-f868-4e1e-9ab2-4fd598f360dd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:30.499762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ad7c8-b765-41fd-83e8-4251c485fcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:30.500832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ad7c8-b765-41fd-83e8-4251c485fcfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:30.501193 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ad7c8-b765-41fd-83e8-4251c485fcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:30.501612 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ad7c8-b765-41fd-83e8-4251c485fcfd admin admin] PATH_INFO: `/v3/projects/bc40e69ef09e4017b43d9eb687c298a9` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:30.503060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ad7c8-b765-41fd-83e8-4251c485fcfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc40e69ef09e4017b43d9eb687c298a9)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:30.505166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ad7c8-b765-41fd-83e8-4251c485fcfd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:30.559338 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-044ad7c8-b765-41fd-83e8-4251c485fcfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc40e69ef09e4017b43d9eb687c298a9', 'request_id': 'req-044ad7c8-b765-41fd-83e8-4251c485fcfd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:30.565844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-044ad7c8-b765-41fd-83e8-4251c485fcfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc40e69ef09e4017b43d9eb687c298a9', 'request_id': 'req-044ad7c8-b765-41fd-83e8-4251c485fcfd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:30.576018 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-044ad7c8-b765-41fd-83e8-4251c485fcfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc40e69ef09e4017b43d9eb687c298a9', 'request_id': 'req-044ad7c8-b765-41fd-83e8-4251c485fcfd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:30.583252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:30.589791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-044ad7c8-b765-41fd-83e8-4251c485fcfd admin admin] The token cache is being invalidate because project bc40e69ef09e4017b43d9eb687c298a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:12:30.590293 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-044ad7c8-b765-41fd-83e8-4251c485fcfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:30.591217 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-044ad7c8-b765-41fd-83e8-4251c485fcfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:30.592799 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1181/2359] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:12:30 2022] DELETE /identity/v3/projects/bc40e69ef09e4017b43d9eb687c298a9 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:12:30.598283 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e95d5e-7c23-46e4-b205-69a85c926893 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:30.598580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e95d5e-7c23-46e4-b205-69a85c926893 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:30.598869 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e95d5e-7c23-46e4-b205-69a85c926893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:30.616051 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-e4e95d5e-7c23-46e4-b205-69a85c926893 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:30.617279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e4e95d5e-7c23-46e4-b205-69a85c926893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:30.677893 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1179/2360] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:30.687387 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b7e44-64c7-48ba-b9af-275431b1b02a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:30.687784 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b7e44-64c7-48ba-b9af-275431b1b02a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:30.688241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b7e44-64c7-48ba-b9af-275431b1b02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:30.707157 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-620b7e44-64c7-48ba-b9af-275431b1b02a None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:30.708273 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-620b7e44-64c7-48ba-b9af-275431b1b02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:30.716374 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1182/2361] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:30.721372 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e95d5e-7c23-46e4-b205-69a85c926893 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:30.731817 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e95d5e-7c23-46e4-b205-69a85c926893 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:30.736389 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf47d2-8605-4298-88c5-bff9d9375164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:30.737078 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf47d2-8605-4298-88c5-bff9d9375164 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:30.737364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf47d2-8605-4298-88c5-bff9d9375164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:30.737642 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf47d2-8605-4298-88c5-bff9d9375164 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:30.738369 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf47d2-8605-4298-88c5-bff9d9375164 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:12:30.738673 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf47d2-8605-4298-88c5-bff9d9375164 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:30.740310 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf47d2-8605-4298-88c5-bff9d9375164 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:30.751874 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1180/2362] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:12:30.756212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b7e44-64c7-48ba-b9af-275431b1b02a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:30.766335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b7e44-64c7-48ba-b9af-275431b1b02a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:30.770929 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b7936-8e43-4d6f-a4c0-16773bf98ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:30.771583 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b7936-8e43-4d6f-a4c0-16773bf98ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:30.771894 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b7936-8e43-4d6f-a4c0-16773bf98ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:30.772278 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b7936-8e43-4d6f-a4c0-16773bf98ab2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:30.772973 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b7936-8e43-4d6f-a4c0-16773bf98ab2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:30.774926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b7936-8e43-4d6f-a4c0-16773bf98ab2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:30.803139 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1183/2363] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:12:30 2022] POST /identity/v3/projects => generated 399 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:12:30.804638 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf47d2-8605-4298-88c5-bff9d9375164 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:30.815478 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf47d2-8605-4298-88c5-bff9d9375164 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:30.845130 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:30.895161 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22c350-cd6d-4ea5-b36a-882c67eddf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:30.896297 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22c350-cd6d-4ea5-b36a-882c67eddf40 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:30.897017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22c350-cd6d-4ea5-b36a-882c67eddf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:30.897573 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22c350-cd6d-4ea5-b36a-882c67eddf40 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:30.898819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22c350-cd6d-4ea5-b36a-882c67eddf40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:30.902165 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22c350-cd6d-4ea5-b36a-882c67eddf40 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:30.932620 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1181/2364] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:12:30 2022] POST /identity/v3/users => generated 300 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:12:30.950081 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b7936-8e43-4d6f-a4c0-16773bf98ab2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:30.969062 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b7936-8e43-4d6f-a4c0-16773bf98ab2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:30.974423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53525a6-638d-4519-ae0c-0296aa6ee300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:30.975199 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53525a6-638d-4519-ae0c-0296aa6ee300 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:30.975977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53525a6-638d-4519-ae0c-0296aa6ee300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:30.975977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53525a6-638d-4519-ae0c-0296aa6ee300 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:30.976805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53525a6-638d-4519-ae0c-0296aa6ee300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:30.978522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53525a6-638d-4519-ae0c-0296aa6ee300 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:30.989448 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1184/2365] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:12:30.993572 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22c350-cd6d-4ea5-b36a-882c67eddf40 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:31.004972 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22c350-cd6d-4ea5-b36a-882c67eddf40 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:31.009961 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d4398-a989-49d8-9eb8-29a7697f1945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:31.010620 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d4398-a989-49d8-9eb8-29a7697f1945 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:31.010908 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d4398-a989-49d8-9eb8-29a7697f1945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:31.011187 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d4398-a989-49d8-9eb8-29a7697f1945 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:31.011937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753d4398-a989-49d8-9eb8-29a7697f1945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:31.013846 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753d4398-a989-49d8-9eb8-29a7697f1945 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:31.026462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1182/2366] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:12:31.030624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53525a6-638d-4519-ae0c-0296aa6ee300 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:31.041281 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53525a6-638d-4519-ae0c-0296aa6ee300 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:31.045975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a464d-1db9-4ea1-8f85-aba06816f165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:31.046897 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a464d-1db9-4ea1-8f85-aba06816f165 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:31.047253 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a464d-1db9-4ea1-8f85-aba06816f165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:31.047659 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a464d-1db9-4ea1-8f85-aba06816f165 admin admin] PATH_INFO: `/v3/projects/989fc483a6b14618899de4dfc2e61535/users/1954b93216fe4d8a84f6ecc351882f52/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:31.064912 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a464d-1db9-4ea1-8f85-aba06816f165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=989fc483a6b14618899de4dfc2e61535, user_id=1954b93216fe4d8a84f6ecc351882f52, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:31.067569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a464d-1db9-4ea1-8f85-aba06816f165 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:31.086015 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1185/2367] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:12:31 2022] PUT /identity/v3/projects/989fc483a6b14618899de4dfc2e61535/users/1954b93216fe4d8a84f6ecc351882f52/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:12:31.152937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bef987-0bb7-4b54-b93c-dfde890143b9 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:31.153234 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bef987-0bb7-4b54-b93c-dfde890143b9 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:31.153522 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bef987-0bb7-4b54-b93c-dfde890143b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:31.180088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-a8bef987-0bb7-4b54-b93c-dfde890143b9 None None] MFA Rules not processed for user `1954b93216fe4d8a84f6ecc351882f52`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:31.181317 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a8bef987-0bb7-4b54-b93c-dfde890143b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:31.253515 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1183/2368] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:12:31 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:31.264842 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a464d-1db9-4ea1-8f85-aba06816f165 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:31.266566 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ae1a464d-1db9-4ea1-8f85-aba06816f165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:31.325486 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a464d-1db9-4ea1-8f85-aba06816f165 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:31.329598 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e990d-fcd8-4150-9a7b-91361971e9e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:31.330264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e990d-fcd8-4150-9a7b-91361971e9e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:31.330546 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e990d-fcd8-4150-9a7b-91361971e9e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:31.330820 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e990d-fcd8-4150-9a7b-91361971e9e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:31.340033 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e990d-fcd8-4150-9a7b-91361971e9e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:31.341546 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e990d-fcd8-4150-9a7b-91361971e9e9 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:31.350716 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1186/2369] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:12:31 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:12:41.249460 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=55,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:41.346430 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:44.796916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bef987-0bb7-4b54-b93c-dfde890143b9 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:44.798869 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a8bef987-0bb7-4b54-b93c-dfde890143b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:44.855576 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bef987-0bb7-4b54-b93c-dfde890143b9 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:44.859246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06b8bb-fc2f-4573-883c-d42798ddf686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:44.859832 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06b8bb-fc2f-4573-883c-d42798ddf686 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:44.860167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06b8bb-fc2f-4573-883c-d42798ddf686 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:44.860467 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06b8bb-fc2f-4573-883c-d42798ddf686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:44.863496 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3c06b8bb-fc2f-4573-883c-d42798ddf686 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:44.925964 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c06b8bb-fc2f-4573-883c-d42798ddf686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:44.927424 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c06b8bb-fc2f-4573-883c-d42798ddf686 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:44.937575 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1184/2370] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:12:44 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:12:46.863641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e990d-fcd8-4150-9a7b-91361971e9e9 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:46.952244 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e990d-fcd8-4150-9a7b-91361971e9e9 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:46.957540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c75ff-bd47-495f-9085-463c33c802fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:46.958422 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c75ff-bd47-495f-9085-463c33c802fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:46.958716 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c75ff-bd47-495f-9085-463c33c802fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:46.958999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c75ff-bd47-495f-9085-463c33c802fd admin admin] PATH_INFO: `/v3/users/1954b93216fe4d8a84f6ecc351882f52` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:46.960259 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3c75ff-bd47-495f-9085-463c33c802fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1954b93216fe4d8a84f6ecc351882f52)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:46.962205 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3c75ff-bd47-495f-9085-463c33c802fd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:47.013843 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6d3c75ff-bd47-495f-9085-463c33c802fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1954b93216fe4d8a84f6ecc351882f52'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:47.022442 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6d3c75ff-bd47-495f-9085-463c33c802fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1954b93216fe4d8a84f6ecc351882f52'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:47.031229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6d3c75ff-bd47-495f-9085-463c33c802fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1954b93216fe4d8a84f6ecc351882f52'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:47.032039 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6d3c75ff-bd47-495f-9085-463c33c802fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1954b93216fe4d8a84f6ecc351882f52'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:47.047901 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6d3c75ff-bd47-495f-9085-463c33c802fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1954b93216fe4d8a84f6ecc351882f52'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:47.054255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1187/2371] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:12:46 2022] DELETE /identity/v3/users/1954b93216fe4d8a84f6ecc351882f52 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:12:47.058985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b104856-f76a-47e5-87e2-4ec1b6b8ec27 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:47.059848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b104856-f76a-47e5-87e2-4ec1b6b8ec27 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:47.060433 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b104856-f76a-47e5-87e2-4ec1b6b8ec27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:47.077781 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-4b104856-f76a-47e5-87e2-4ec1b6b8ec27 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:47.079025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4b104856-f76a-47e5-87e2-4ec1b6b8ec27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:47.140248 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1185/2372] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:12:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:12:47.150606 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c75ff-bd47-495f-9085-463c33c802fd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:47.152132 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6d3c75ff-bd47-495f-9085-463c33c802fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:47.211660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c75ff-bd47-495f-9085-463c33c802fd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:47.215547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7f0d5-ef04-457f-a683-2142b7f0f782 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:47.216261 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7f0d5-ef04-457f-a683-2142b7f0f782 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:47.216548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7f0d5-ef04-457f-a683-2142b7f0f782 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:47.216824 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7f0d5-ef04-457f-a683-2142b7f0f782 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:47.227176 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7f0d5-ef04-457f-a683-2142b7f0f782 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:47.228689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7f0d5-ef04-457f-a683-2142b7f0f782 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:47.240033 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1188/2373] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:12:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:12:47.670857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b104856-f76a-47e5-87e2-4ec1b6b8ec27 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:47.672754 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4b104856-f76a-47e5-87e2-4ec1b6b8ec27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:47.693632 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b104856-f76a-47e5-87e2-4ec1b6b8ec27 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:47.698450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20dbae-b7cb-422f-afdb-6732cb58ac65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:47.699349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20dbae-b7cb-422f-afdb-6732cb58ac65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:47.699635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20dbae-b7cb-422f-afdb-6732cb58ac65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:47.699914 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20dbae-b7cb-422f-afdb-6732cb58ac65 admin admin] PATH_INFO: `/v3/projects/989fc483a6b14618899de4dfc2e61535` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:47.701392 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f20dbae-b7cb-422f-afdb-6732cb58ac65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=989fc483a6b14618899de4dfc2e61535)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:47.703384 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f20dbae-b7cb-422f-afdb-6732cb58ac65 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:47.753207 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:47.756479 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4f20dbae-b7cb-422f-afdb-6732cb58ac65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '989fc483a6b14618899de4dfc2e61535', 'request_id': 'req-4f20dbae-b7cb-422f-afdb-6732cb58ac65'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:47.763196 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4f20dbae-b7cb-422f-afdb-6732cb58ac65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '989fc483a6b14618899de4dfc2e61535', 'request_id': 'req-4f20dbae-b7cb-422f-afdb-6732cb58ac65'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:47.771255 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4f20dbae-b7cb-422f-afdb-6732cb58ac65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '989fc483a6b14618899de4dfc2e61535', 'request_id': 'req-4f20dbae-b7cb-422f-afdb-6732cb58ac65'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:47.783796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4f20dbae-b7cb-422f-afdb-6732cb58ac65 admin admin] The token cache is being invalidate because project 989fc483a6b14618899de4dfc2e61535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:12:47.784130 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4f20dbae-b7cb-422f-afdb-6732cb58ac65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:47.784803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4f20dbae-b7cb-422f-afdb-6732cb58ac65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:12:47.786603 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1186/2374] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:12:47 2022] DELETE /identity/v3/projects/989fc483a6b14618899de4dfc2e61535 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:12:57.234841 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:57.780411 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:33.063777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20c0dd-3cc3-4ada-b3b7-f4704342ae08 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:33.064347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20c0dd-3cc3-4ada-b3b7-f4704342ae08 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:33.064347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20c0dd-3cc3-4ada-b3b7-f4704342ae08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:33.081565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ed20c0dd-3cc3-4ada-b3b7-f4704342ae08 None None] MFA Rules not processed for user `0542cd47ba77454ba25170292ed9aeb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:13:33.082771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ed20c0dd-3cc3-4ada-b3b7-f4704342ae08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:33.138973 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1189/2375] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:13:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:13:33.179859 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20dbae-b7cb-422f-afdb-6732cb58ac65 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:13:33.182139 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4f20dbae-b7cb-422f-afdb-6732cb58ac65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:33.242874 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20dbae-b7cb-422f-afdb-6732cb58ac65 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:13:33.246357 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea2557-63b7-4e8d-b829-47ce31f032a7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '503a8228d4754c309fc3f365f4cfc61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:13:33.247006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea2557-63b7-4e8d-b829-47ce31f032a7 service glance] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:33.247296 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea2557-63b7-4e8d-b829-47ce31f032a7 service glance] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:33.247575 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea2557-63b7-4e8d-b829-47ce31f032a7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:33.249941 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d6ea2557-63b7-4e8d-b829-47ce31f032a7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:33.330370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea2557-63b7-4e8d-b829-47ce31f032a7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:13:33.331826 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea2557-63b7-4e8d-b829-47ce31f032a7 service glance] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:13:33.343904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1187/2376] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:13:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:13:33.981287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20c0dd-3cc3-4ada-b3b7-f4704342ae08 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:13:33.983425 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ed20c0dd-3cc3-4ada-b3b7-f4704342ae08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:34.063052 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20c0dd-3cc3-4ada-b3b7-f4704342ae08 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:13:34.066628 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843fcff4-b5c8-4c90-98c8-a4c49db5a9ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:13:34.067278 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fcff4-b5c8-4c90-98c8-a4c49db5a9ae service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:34.067568 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fcff4-b5c8-4c90-98c8-a4c49db5a9ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:34.067849 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fcff4-b5c8-4c90-98c8-a4c49db5a9ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:34.069651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-843fcff4-b5c8-4c90-98c8-a4c49db5a9ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:34.083846 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843fcff4-b5c8-4c90-98c8-a4c49db5a9ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:13:34.085613 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843fcff4-b5c8-4c90-98c8-a4c49db5a9ae service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:13:34.097636 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1190/2377] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:13:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:13:36.135533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea2557-63b7-4e8d-b829-47ce31f032a7 service glance] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:13:36.137344 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d6ea2557-63b7-4e8d-b829-47ce31f032a7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:36.214919 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea2557-63b7-4e8d-b829-47ce31f032a7 service glance] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:13:36.219833 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f000c4-f299-45ad-8851-05dd23c40d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:13:36.220824 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f000c4-f299-45ad-8851-05dd23c40d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:36.221115 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f000c4-f299-45ad-8851-05dd23c40d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:36.221400 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f000c4-f299-45ad-8851-05dd23c40d8d admin admin] PATH_INFO: `/v3/users/67814b29011342ba90e603a1d389bea4` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:36.222620 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f000c4-f299-45ad-8851-05dd23c40d8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=67814b29011342ba90e603a1d389bea4)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:13:36.224741 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f000c4-f299-45ad-8851-05dd23c40d8d admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:13:36.281747 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:36.286440 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d6f000c4-f299-45ad-8851-05dd23c40d8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67814b29011342ba90e603a1d389bea4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:13:36.296783 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d6f000c4-f299-45ad-8851-05dd23c40d8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67814b29011342ba90e603a1d389bea4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:13:36.304550 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d6f000c4-f299-45ad-8851-05dd23c40d8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67814b29011342ba90e603a1d389bea4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:13:36.305228 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d6f000c4-f299-45ad-8851-05dd23c40d8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67814b29011342ba90e603a1d389bea4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:13:36.318776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d6f000c4-f299-45ad-8851-05dd23c40d8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67814b29011342ba90e603a1d389bea4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:13:36.325215 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1188/2378] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:13:36 2022] DELETE /identity/v3/users/67814b29011342ba90e603a1d389bea4 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:13:36.329547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74c33c-f505-458f-8f47-aff67cf0be3d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:36.330034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74c33c-f505-458f-8f47-aff67cf0be3d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:36.330159 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74c33c-f505-458f-8f47-aff67cf0be3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:36.362498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2a74c33c-f505-458f-8f47-aff67cf0be3d None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:13:36.363980 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2a74c33c-f505-458f-8f47-aff67cf0be3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:36.430973 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1191/2379] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:13:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:13:36.441031 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f000c4-f299-45ad-8851-05dd23c40d8d admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:13:36.442649 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d6f000c4-f299-45ad-8851-05dd23c40d8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:36.506547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f000c4-f299-45ad-8851-05dd23c40d8d admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:13:36.510568 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0bae9d-e177-4c17-b8f3-a233d1c8e187 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:13:36.511235 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bae9d-e177-4c17-b8f3-a233d1c8e187 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:36.511518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bae9d-e177-4c17-b8f3-a233d1c8e187 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:36.511804 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0bae9d-e177-4c17-b8f3-a233d1c8e187 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:36.531864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0bae9d-e177-4c17-b8f3-a233d1c8e187 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:13:36.533222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0bae9d-e177-4c17-b8f3-a233d1c8e187 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:13:36.544456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1189/2380] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:13:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:13:36.845628 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74c33c-f505-458f-8f47-aff67cf0be3d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:13:36.847408 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2a74c33c-f505-458f-8f47-aff67cf0be3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:36.864776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74c33c-f505-458f-8f47-aff67cf0be3d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:13:36.872646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcedb36-ba6c-42bf-97a5-d8057a38ba65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:13:36.874370 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcedb36-ba6c-42bf-97a5-d8057a38ba65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:36.875056 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcedb36-ba6c-42bf-97a5-d8057a38ba65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:36.875532 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcedb36-ba6c-42bf-97a5-d8057a38ba65 admin admin] PATH_INFO: `/v3/projects/cb3c985bd0514ed098260ca5b836ae66` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:36.877097 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcedb36-ba6c-42bf-97a5-d8057a38ba65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb3c985bd0514ed098260ca5b836ae66)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:13:36.879283 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcedb36-ba6c-42bf-97a5-d8057a38ba65 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:13:36.977100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:36.982536 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ebcedb36-ba6c-42bf-97a5-d8057a38ba65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb3c985bd0514ed098260ca5b836ae66', 'request_id': 'req-ebcedb36-ba6c-42bf-97a5-d8057a38ba65'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:13:36.991877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ebcedb36-ba6c-42bf-97a5-d8057a38ba65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb3c985bd0514ed098260ca5b836ae66', 'request_id': 'req-ebcedb36-ba6c-42bf-97a5-d8057a38ba65'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:13:37.001624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ebcedb36-ba6c-42bf-97a5-d8057a38ba65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb3c985bd0514ed098260ca5b836ae66', 'request_id': 'req-ebcedb36-ba6c-42bf-97a5-d8057a38ba65'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:13:37.013354 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ebcedb36-ba6c-42bf-97a5-d8057a38ba65 admin admin] The token cache is being invalidate because project cb3c985bd0514ed098260ca5b836ae66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:13:37.013654 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ebcedb36-ba6c-42bf-97a5-d8057a38ba65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:13:37.014469 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ebcedb36-ba6c-42bf-97a5-d8057a38ba65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:13:37.016810 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1192/2381] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:13:36 2022] DELETE /identity/v3/projects/cb3c985bd0514ed098260ca5b836ae66 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:13:37.539224 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0bae9d-e177-4c17-b8f3-a233d1c8e187 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:13:37.540806 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-cd0bae9d-e177-4c17-b8f3-a233d1c8e187 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:37.614682 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0bae9d-e177-4c17-b8f3-a233d1c8e187 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:13:37.666590 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:37.677776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143ebd3-aa99-4665-9288-08472cdaa8be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:13:37.678421 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143ebd3-aa99-4665-9288-08472cdaa8be service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:37.678708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143ebd3-aa99-4665-9288-08472cdaa8be service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:37.678986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143ebd3-aa99-4665-9288-08472cdaa8be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:37.680780 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1143ebd3-aa99-4665-9288-08472cdaa8be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:37.738459 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143ebd3-aa99-4665-9288-08472cdaa8be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:13:37.740149 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143ebd3-aa99-4665-9288-08472cdaa8be service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:13:37.749234 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1190/2382] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:13:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:13:44.153789 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcedb36-ba6c-42bf-97a5-d8057a38ba65 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:13:44.155880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ebcedb36-ba6c-42bf-97a5-d8057a38ba65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:44.173817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcedb36-ba6c-42bf-97a5-d8057a38ba65 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:13:44.179491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a9ffd-ce22-4e2c-933b-8c7a7a9bf1b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:13:44.180176 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a9ffd-ce22-4e2c-933b-8c7a7a9bf1b9 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:44.180469 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a9ffd-ce22-4e2c-933b-8c7a7a9bf1b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:44.181014 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a9ffd-ce22-4e2c-933b-8c7a7a9bf1b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:44.182853 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-820a9ffd-ce22-4e2c-933b-8c7a7a9bf1b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:44.254293 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a9ffd-ce22-4e2c-933b-8c7a7a9bf1b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:13:44.255910 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a9ffd-ce22-4e2c-933b-8c7a7a9bf1b9 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:13:44.264943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1193/2383] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:13:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:13:47.745191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=28 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:54.260833 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=36,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:58.991461 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143ebd3-aa99-4665-9288-08472cdaa8be service nova] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:13:58.992933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1143ebd3-aa99-4665-9288-08472cdaa8be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:59.050328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143ebd3-aa99-4665-9288-08472cdaa8be service nova] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:13:59.053947 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554c64c-42ef-4eea-bdad-190affca0369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:13:59.054598 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554c64c-42ef-4eea-bdad-190affca0369 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:59.054887 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554c64c-42ef-4eea-bdad-190affca0369 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:59.055436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554c64c-42ef-4eea-bdad-190affca0369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:59.057187 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c554c64c-42ef-4eea-bdad-190affca0369 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:59.067130 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c554c64c-42ef-4eea-bdad-190affca0369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:13:59.068594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c554c64c-42ef-4eea-bdad-190affca0369 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:13:59.076595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1191/2384] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:13:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:14:09.072980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=26 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:14:15.442096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a9ffd-ce22-4e2c-933b-8c7a7a9bf1b9 service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:14:15.443542 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-820a9ffd-ce22-4e2c-933b-8c7a7a9bf1b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:14:15.493481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554c64c-42ef-4eea-bdad-190affca0369 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:14:15.495790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c554c64c-42ef-4eea-bdad-190affca0369 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:14:15.528200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a9ffd-ce22-4e2c-933b-8c7a7a9bf1b9 service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:14:15.533316 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e296080-c0ad-40fb-b7fd-a2d1ec23896d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd5b2ba9e37041c78a99f1700a4970c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:14:15.533969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e296080-c0ad-40fb-b7fd-a2d1ec23896d service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:15.534259 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e296080-c0ad-40fb-b7fd-a2d1ec23896d service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:15.534542 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e296080-c0ad-40fb-b7fd-a2d1ec23896d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:15.536430 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9e296080-c0ad-40fb-b7fd-a2d1ec23896d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:14:15.548574 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e296080-c0ad-40fb-b7fd-a2d1ec23896d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:14:15.550186 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e296080-c0ad-40fb-b7fd-a2d1ec23896d service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:14:15.561198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1194/2385] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:14:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:14:15.575082 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554c64c-42ef-4eea-bdad-190affca0369 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:14:15.579900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b0c70-e83d-4e5a-9503-e7d36a3c5041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:14:15.580806 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b0c70-e83d-4e5a-9503-e7d36a3c5041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:15.581096 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b0c70-e83d-4e5a-9503-e7d36a3c5041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:15.581379 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b0c70-e83d-4e5a-9503-e7d36a3c5041 admin admin] PATH_INFO: `/v3/users/da8ecff90e714e4e961f8a8104143f62` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:15.582604 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b0c70-e83d-4e5a-9503-e7d36a3c5041 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da8ecff90e714e4e961f8a8104143f62)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:14:15.584564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b0c70-e83d-4e5a-9503-e7d36a3c5041 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:14:15.634371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8c7b0c70-e83d-4e5a-9503-e7d36a3c5041 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da8ecff90e714e4e961f8a8104143f62'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:14:15.642032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8c7b0c70-e83d-4e5a-9503-e7d36a3c5041 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da8ecff90e714e4e961f8a8104143f62'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:14:15.648908 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8c7b0c70-e83d-4e5a-9503-e7d36a3c5041 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da8ecff90e714e4e961f8a8104143f62'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:14:15.649544 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8c7b0c70-e83d-4e5a-9503-e7d36a3c5041 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da8ecff90e714e4e961f8a8104143f62'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:14:15.661155 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-8c7b0c70-e83d-4e5a-9503-e7d36a3c5041 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da8ecff90e714e4e961f8a8104143f62'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:14:15.667243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1192/2386] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:14:15 2022] DELETE /identity/v3/users/da8ecff90e714e4e961f8a8104143f62 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:14:15.836777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e296080-c0ad-40fb-b7fd-a2d1ec23896d service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:14:15.848816 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e296080-c0ad-40fb-b7fd-a2d1ec23896d service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:14:15.905663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f584dbf-55b2-490c-ae21-2be3c139f5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:14:15.906554 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f584dbf-55b2-490c-ae21-2be3c139f5dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:15.906924 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f584dbf-55b2-490c-ae21-2be3c139f5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:15.907205 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f584dbf-55b2-490c-ae21-2be3c139f5dd admin admin] PATH_INFO: `/v3/projects/9f6e581c160d4e64a81e0b35907cc078` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:15.908561 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f584dbf-55b2-490c-ae21-2be3c139f5dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f6e581c160d4e64a81e0b35907cc078)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:14:15.910462 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f584dbf-55b2-490c-ae21-2be3c139f5dd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:14:15.961532 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6f584dbf-55b2-490c-ae21-2be3c139f5dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f6e581c160d4e64a81e0b35907cc078', 'request_id': 'req-6f584dbf-55b2-490c-ae21-2be3c139f5dd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:14:15.967557 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6f584dbf-55b2-490c-ae21-2be3c139f5dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f6e581c160d4e64a81e0b35907cc078', 'request_id': 'req-6f584dbf-55b2-490c-ae21-2be3c139f5dd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:14:15.974883 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6f584dbf-55b2-490c-ae21-2be3c139f5dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f6e581c160d4e64a81e0b35907cc078', 'request_id': 'req-6f584dbf-55b2-490c-ae21-2be3c139f5dd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:14:15.985776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6f584dbf-55b2-490c-ae21-2be3c139f5dd admin admin] The token cache is being invalidate because project 9f6e581c160d4e64a81e0b35907cc078 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:14:15.986060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6f584dbf-55b2-490c-ae21-2be3c139f5dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:14:15.986680 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6f584dbf-55b2-490c-ae21-2be3c139f5dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:14:15.988240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1195/2387] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:14:15 2022] DELETE /identity/v3/projects/9f6e581c160d4e64a81e0b35907cc078 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:14:15.993339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c51b1e-6a23-4d32-826e-ef45097a5cbc None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:15.993641 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c51b1e-6a23-4d32-826e-ef45097a5cbc None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:15.993929 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c51b1e-6a23-4d32-826e-ef45097a5cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:16.012739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-a4c51b1e-6a23-4d32-826e-ef45097a5cbc None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:14:16.013881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a4c51b1e-6a23-4d32-826e-ef45097a5cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:14:16.073665 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1193/2388] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:14:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:14:16.081670 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33aadd0-eb08-42f4-b68a-8e96802336e2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:16.081968 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33aadd0-eb08-42f4-b68a-8e96802336e2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:16.082287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33aadd0-eb08-42f4-b68a-8e96802336e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:16.100226 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f33aadd0-eb08-42f4-b68a-8e96802336e2 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:14:16.101429 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f33aadd0-eb08-42f4-b68a-8e96802336e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:14:16.109335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1196/2389] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:14:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:14:16.113801 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c51b1e-6a23-4d32-826e-ef45097a5cbc None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:14:16.123345 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c51b1e-6a23-4d32-826e-ef45097a5cbc None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:14:16.128472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210bbca-a6af-4a0e-921f-2c7f83ad5d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:14:16.129117 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210bbca-a6af-4a0e-921f-2c7f83ad5d06 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:16.129407 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210bbca-a6af-4a0e-921f-2c7f83ad5d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:16.129691 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210bbca-a6af-4a0e-921f-2c7f83ad5d06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:16.130420 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210bbca-a6af-4a0e-921f-2c7f83ad5d06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:14:16.130733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210bbca-a6af-4a0e-921f-2c7f83ad5d06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:14:16.132390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210bbca-a6af-4a0e-921f-2c7f83ad5d06 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:14:16.143767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1194/2390] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:14:16.147616 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33aadd0-eb08-42f4-b68a-8e96802336e2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:14:16.156958 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33aadd0-eb08-42f4-b68a-8e96802336e2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:14:16.161689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2afa98-7116-4a21-bd06-2e615082a4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:14:16.162336 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2afa98-7116-4a21-bd06-2e615082a4ac admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:16.162630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2afa98-7116-4a21-bd06-2e615082a4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:16.162915 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2afa98-7116-4a21-bd06-2e615082a4ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:16.163620 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2afa98-7116-4a21-bd06-2e615082a4ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:14:16.165578 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2afa98-7116-4a21-bd06-2e615082a4ac admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:14:16.176005 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:14:16.194170 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1197/2391] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:14:16 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:14:16.198190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210bbca-a6af-4a0e-921f-2c7f83ad5d06 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:14:16.208188 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210bbca-a6af-4a0e-921f-2c7f83ad5d06 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:14:16.211702 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:14:16.268246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bcefe1-0557-41fe-a66b-93a6ce9262f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:14:16.268843 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bcefe1-0557-41fe-a66b-93a6ce9262f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:16.269135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bcefe1-0557-41fe-a66b-93a6ce9262f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:16.269419 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bcefe1-0557-41fe-a66b-93a6ce9262f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:16.270110 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bcefe1-0557-41fe-a66b-93a6ce9262f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:14:16.272077 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bcefe1-0557-41fe-a66b-93a6ce9262f3 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:14:16.291636 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1195/2392] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:14:16 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:14:16.295903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2afa98-7116-4a21-bd06-2e615082a4ac admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:14:16.306814 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2afa98-7116-4a21-bd06-2e615082a4ac admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:14:16.311402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd78c1e-a0aa-4ffc-a460-d78e8a08584e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:14:16.312087 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd78c1e-a0aa-4ffc-a460-d78e8a08584e admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:16.312374 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd78c1e-a0aa-4ffc-a460-d78e8a08584e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:16.312649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd78c1e-a0aa-4ffc-a460-d78e8a08584e admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:16.313924 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd78c1e-a0aa-4ffc-a460-d78e8a08584e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:14:16.315504 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd78c1e-a0aa-4ffc-a460-d78e8a08584e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:14:16.325782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1198/2393] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:14:16.329697 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bcefe1-0557-41fe-a66b-93a6ce9262f3 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:14:16.339366 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bcefe1-0557-41fe-a66b-93a6ce9262f3 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:14:16.343986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4481e0-8112-42fd-b1cd-d05f682887ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:14:16.345169 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4481e0-8112-42fd-b1cd-d05f682887ed admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:16.345456 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4481e0-8112-42fd-b1cd-d05f682887ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:16.345729 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4481e0-8112-42fd-b1cd-d05f682887ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:16.346441 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4481e0-8112-42fd-b1cd-d05f682887ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:14:16.348037 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4481e0-8112-42fd-b1cd-d05f682887ed admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:14:16.357638 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1196/2394] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:14:16.362127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd78c1e-a0aa-4ffc-a460-d78e8a08584e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:14:16.372093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd78c1e-a0aa-4ffc-a460-d78e8a08584e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:14:16.376754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed668a-6e2c-4a64-8efe-cd5fafb165d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:14:16.377619 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed668a-6e2c-4a64-8efe-cd5fafb165d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:16.377905 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed668a-6e2c-4a64-8efe-cd5fafb165d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:16.378184 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed668a-6e2c-4a64-8efe-cd5fafb165d4 admin admin] PATH_INFO: `/v3/projects/9df5aea1d29c4a91b82a956e152bd214/users/35334a0f92254b5e900aad8c51873809/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:16.393552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed668a-6e2c-4a64-8efe-cd5fafb165d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df5aea1d29c4a91b82a956e152bd214, user_id=35334a0f92254b5e900aad8c51873809, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:14:16.396459 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed668a-6e2c-4a64-8efe-cd5fafb165d4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:14:16.409766 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1199/2395] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:14:16 2022] PUT /identity/v3/projects/9df5aea1d29c4a91b82a956e152bd214/users/35334a0f92254b5e900aad8c51873809/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:14:16.415157 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77709c80-00e2-4ee9-992c-db76b63bb65d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:16.415450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77709c80-00e2-4ee9-992c-db76b63bb65d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:16.415730 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77709c80-00e2-4ee9-992c-db76b63bb65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:16.433951 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-77709c80-00e2-4ee9-992c-db76b63bb65d None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:14:16.435142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-77709c80-00e2-4ee9-992c-db76b63bb65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:14:16.491441 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1197/2396] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:14:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:14:16.501086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed668a-6e2c-4a64-8efe-cd5fafb165d4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:14:16.502832 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c2ed668a-6e2c-4a64-8efe-cd5fafb165d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:14:16.558980 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed668a-6e2c-4a64-8efe-cd5fafb165d4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:14:16.562555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f3f12-af0a-4884-85cc-86830b9ef151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:14:16.563225 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f3f12-af0a-4884-85cc-86830b9ef151 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:16.563513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f3f12-af0a-4884-85cc-86830b9ef151 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:16.563794 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f3f12-af0a-4884-85cc-86830b9ef151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:16.572675 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f3f12-af0a-4884-85cc-86830b9ef151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:14:16.574103 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f3f12-af0a-4884-85cc-86830b9ef151 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:14:16.583454 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1200/2397] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:14:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:14:20.388467 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ebf51-8c5a-4d6f-8fcf-b800bb0bb7ae None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:20.388984 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ebf51-8c5a-4d6f-8fcf-b800bb0bb7ae None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:20.389095 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ebf51-8c5a-4d6f-8fcf-b800bb0bb7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:20.421840 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-4c7ebf51-8c5a-4d6f-8fcf-b800bb0bb7ae None None] MFA Rules not processed for user `35334a0f92254b5e900aad8c51873809`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:14:20.423626 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4c7ebf51-8c5a-4d6f-8fcf-b800bb0bb7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:14:20.494682 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1198/2398] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:14:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:14:20.499169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f3f12-af0a-4884-85cc-86830b9ef151 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:14:20.509631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f3f12-af0a-4884-85cc-86830b9ef151 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:14:20.514822 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a597f0d-9da1-4fca-b242-98a07cfd596b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:14:20.515497 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a597f0d-9da1-4fca-b242-98a07cfd596b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:20.515857 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a597f0d-9da1-4fca-b242-98a07cfd596b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:20.516162 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a597f0d-9da1-4fca-b242-98a07cfd596b admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:20.516869 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a597f0d-9da1-4fca-b242-98a07cfd596b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:14:20.518809 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a597f0d-9da1-4fca-b242-98a07cfd596b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:14:20.568669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1201/2399] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:14:20 2022] POST /identity/v3/projects => generated 375 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:14:20.572706 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ebf51-8c5a-4d6f-8fcf-b800bb0bb7ae None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:14:20.583538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ebf51-8c5a-4d6f-8fcf-b800bb0bb7ae None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:14:20.594685 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:14:20.649861 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46488dee-c5e4-49fb-8f55-4362d0251c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:14:20.650551 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46488dee-c5e4-49fb-8f55-4362d0251c39 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:20.650843 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46488dee-c5e4-49fb-8f55-4362d0251c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:20.651125 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46488dee-c5e4-49fb-8f55-4362d0251c39 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:20.651833 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46488dee-c5e4-49fb-8f55-4362d0251c39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:14:20.653808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46488dee-c5e4-49fb-8f55-4362d0251c39 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:14:20.675427 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1199/2400] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:14:20 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:14:20.679744 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a597f0d-9da1-4fca-b242-98a07cfd596b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:14:20.692949 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a597f0d-9da1-4fca-b242-98a07cfd596b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:14:20.698226 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28666c3-86c6-45d7-ae1d-6062a9f3fdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:14:20.698911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28666c3-86c6-45d7-ae1d-6062a9f3fdf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:20.699203 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28666c3-86c6-45d7-ae1d-6062a9f3fdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:20.699490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28666c3-86c6-45d7-ae1d-6062a9f3fdf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:20.700270 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28666c3-86c6-45d7-ae1d-6062a9f3fdf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:14:20.701904 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28666c3-86c6-45d7-ae1d-6062a9f3fdf3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:14:20.713931 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1202/2401] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:14:20.717930 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46488dee-c5e4-49fb-8f55-4362d0251c39 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:14:20.728450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46488dee-c5e4-49fb-8f55-4362d0251c39 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:14:20.733276 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681000f0-3009-4093-bea5-bbe34633a2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:14:20.734190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681000f0-3009-4093-bea5-bbe34633a2c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:20.734484 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681000f0-3009-4093-bea5-bbe34633a2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:20.735422 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681000f0-3009-4093-bea5-bbe34633a2c6 admin admin] PATH_INFO: `/v3/projects/9ce4b3331cb847c4a360c4da862dca29/users/7a5c06f51c99434d812700d710ebccaf/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:20.753701 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681000f0-3009-4093-bea5-bbe34633a2c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ce4b3331cb847c4a360c4da862dca29, user_id=7a5c06f51c99434d812700d710ebccaf, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:14:20.756529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681000f0-3009-4093-bea5-bbe34633a2c6 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:14:20.773254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1200/2402] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:14:20 2022] PUT /identity/v3/projects/9ce4b3331cb847c4a360c4da862dca29/users/7a5c06f51c99434d812700d710ebccaf/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:14:20.778376 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28666c3-86c6-45d7-ae1d-6062a9f3fdf3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:14:20.792264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28666c3-86c6-45d7-ae1d-6062a9f3fdf3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:14:20.850976 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec8527-01b0-4253-aa59-4a4acb3f741a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:14:20.851660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec8527-01b0-4253-aa59-4a4acb3f741a admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:20.851969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec8527-01b0-4253-aa59-4a4acb3f741a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:20.852263 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec8527-01b0-4253-aa59-4a4acb3f741a admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:20.853001 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec8527-01b0-4253-aa59-4a4acb3f741a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:14:20.854613 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec8527-01b0-4253-aa59-4a4acb3f741a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:14:20.865803 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1203/2403] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:14:20.869990 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681000f0-3009-4093-bea5-bbe34633a2c6 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:14:20.880613 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681000f0-3009-4093-bea5-bbe34633a2c6 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:14:20.885240 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33579e-595a-43ce-991f-d4ea32f839d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:14:20.886132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33579e-595a-43ce-991f-d4ea32f839d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:20.886422 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33579e-595a-43ce-991f-d4ea32f839d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:20.886706 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33579e-595a-43ce-991f-d4ea32f839d4 admin admin] PATH_INFO: `/v3/projects/9ce4b3331cb847c4a360c4da862dca29/users/7a5c06f51c99434d812700d710ebccaf/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:20.888759 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33579e-595a-43ce-991f-d4ea32f839d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ce4b3331cb847c4a360c4da862dca29, user_id=7a5c06f51c99434d812700d710ebccaf, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:14:20.891443 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33579e-595a-43ce-991f-d4ea32f839d4 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:14:20.902672 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1201/2404] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:14:20 2022] PUT /identity/v3/projects/9ce4b3331cb847c4a360c4da862dca29/users/7a5c06f51c99434d812700d710ebccaf/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:14:25.439266 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96433a49-0f46-4930-ae83-19bf4c553d98 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:25.439734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96433a49-0f46-4930-ae83-19bf4c553d98 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:25.440494 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96433a49-0f46-4930-ae83-19bf4c553d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:25.478364 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-96433a49-0f46-4930-ae83-19bf4c553d98 None None] MFA Rules not processed for user `7a5c06f51c99434d812700d710ebccaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:14:25.480042 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-96433a49-0f46-4930-ae83-19bf4c553d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:14:25.491394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:14:25.565176 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1204/2405] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:14:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:14:25.578812 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33579e-595a-43ce-991f-d4ea32f839d4 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:14:25.580856 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5a33579e-595a-43ce-991f-d4ea32f839d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:14:25.686170 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33579e-595a-43ce-991f-d4ea32f839d4 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:14:25.692687 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa419f12-4e28-4f23-a0ab-5fefec60289e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:14:25.693060 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa419f12-4e28-4f23-a0ab-5fefec60289e service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:25.693461 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa419f12-4e28-4f23-a0ab-5fefec60289e service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:25.693827 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa419f12-4e28-4f23-a0ab-5fefec60289e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:25.759998 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa419f12-4e28-4f23-a0ab-5fefec60289e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:14:25.761617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa419f12-4e28-4f23-a0ab-5fefec60289e service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:14:25.772441 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1202/2406] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:14:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:14:35.562690 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=24 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:14:35.768280 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=93,keystone:INSERT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:14:37.877100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96433a49-0f46-4930-ae83-19bf4c553d98 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:14:37.878620 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-96433a49-0f46-4930-ae83-19bf4c553d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:14:37.894436 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96433a49-0f46-4930-ae83-19bf4c553d98 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:14:37.899810 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114112c-6501-490b-b5b7-10864112861f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:14:37.900481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114112c-6501-490b-b5b7-10864112861f service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:37.900771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114112c-6501-490b-b5b7-10864112861f service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:37.901052 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114112c-6501-490b-b5b7-10864112861f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:37.902746 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b114112c-6501-490b-b5b7-10864112861f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:14:37.961097 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114112c-6501-490b-b5b7-10864112861f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:14:37.962773 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114112c-6501-490b-b5b7-10864112861f service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:14:37.972592 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1205/2407] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:14:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:14:47.967631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=28 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:14:53.537250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f206d0-9bd2-41ca-8a31-e8e5313c7d2d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:53.537794 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f206d0-9bd2-41ca-8a31-e8e5313c7d2d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:53.537794 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f206d0-9bd2-41ca-8a31-e8e5313c7d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:53.555304 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c8f206d0-9bd2-41ca-8a31-e8e5313c7d2d None None] MFA Rules not processed for user `35334a0f92254b5e900aad8c51873809`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:14:53.556530 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c8f206d0-9bd2-41ca-8a31-e8e5313c7d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:14:53.563861 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1203/2408] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:14:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:14:53.888593 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114112c-6501-490b-b5b7-10864112861f service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:14:53.950362 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114112c-6501-490b-b5b7-10864112861f service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:14:53.954521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2dc0a-c09a-430f-834c-346a44ca116b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:14:53.955171 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2dc0a-c09a-430f-834c-346a44ca116b service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:53.955454 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2dc0a-c09a-430f-834c-346a44ca116b service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:53.955734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2dc0a-c09a-430f-834c-346a44ca116b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:53.957597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-22e2dc0a-c09a-430f-834c-346a44ca116b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:14:53.968220 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e2dc0a-c09a-430f-834c-346a44ca116b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:14:53.969765 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e2dc0a-c09a-430f-834c-346a44ca116b service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:14:53.978213 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1206/2409] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:14:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:14:55.486581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f206d0-9bd2-41ca-8a31-e8e5313c7d2d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:14:55.487999 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c8f206d0-9bd2-41ca-8a31-e8e5313c7d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:14:55.556912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f206d0-9bd2-41ca-8a31-e8e5313c7d2d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:14:55.567525 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba188449-e51a-4575-9a1e-4ba0f6c51792 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd5b2ba9e37041c78a99f1700a4970c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:14:55.568233 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba188449-e51a-4575-9a1e-4ba0f6c51792 service placement] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:14:55.568522 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba188449-e51a-4575-9a1e-4ba0f6c51792 service placement] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:14:55.568805 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba188449-e51a-4575-9a1e-4ba0f6c51792 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:14:55.570632 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ba188449-e51a-4575-9a1e-4ba0f6c51792 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:14:55.585383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba188449-e51a-4575-9a1e-4ba0f6c51792 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:14:55.586987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba188449-e51a-4575-9a1e-4ba0f6c51792 service placement] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:14:55.597950 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1204/2410] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:14:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:15:01.594768 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2dc0a-c09a-430f-834c-346a44ca116b service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:15:01.614078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2dc0a-c09a-430f-834c-346a44ca116b service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:15:01.623449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c66035c-8235-423d-b125-bf0fe4e706bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:15:01.624159 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66035c-8235-423d-b125-bf0fe4e706bd service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:01.624636 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66035c-8235-423d-b125-bf0fe4e706bd service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:01.625105 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66035c-8235-423d-b125-bf0fe4e706bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:01.627009 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5c66035c-8235-423d-b125-bf0fe4e706bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:01.640630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c66035c-8235-423d-b125-bf0fe4e706bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:15:01.642324 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c66035c-8235-423d-b125-bf0fe4e706bd service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:15:01.658552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1207/2411] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:15:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:15:05.593659 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=35 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:15:08.856439 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79237df0-f2f0-43e0-acd5-1cbe43ea0bb5 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:08.857594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79237df0-f2f0-43e0-acd5-1cbe43ea0bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:08.857594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79237df0-f2f0-43e0-acd5-1cbe43ea0bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:08.877094 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-79237df0-f2f0-43e0-acd5-1cbe43ea0bb5 None None] MFA Rules not processed for user `3720c2225b51464abee34d05a69b9db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:15:08.878905 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-79237df0-f2f0-43e0-acd5-1cbe43ea0bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:08.888645 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1205/2412] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:15:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:15:11.653299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=32 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:15:14.679058 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c66035c-8235-423d-b125-bf0fe4e706bd service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:15:14.694789 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c66035c-8235-423d-b125-bf0fe4e706bd service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:15:14.701869 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbbeca0-a34c-41fa-a986-dfcefaa82ce5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:15:14.702552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbeca0-a34c-41fa-a986-dfcefaa82ce5 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:14.702844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbeca0-a34c-41fa-a986-dfcefaa82ce5 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:14.703125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbbeca0-a34c-41fa-a986-dfcefaa82ce5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:14.705157 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bbbbeca0-a34c-41fa-a986-dfcefaa82ce5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:14.722830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbbeca0-a34c-41fa-a986-dfcefaa82ce5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:15:14.724917 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbbeca0-a34c-41fa-a986-dfcefaa82ce5 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:15:14.738781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1208/2413] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:15:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:15:17.579962 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79237df0-f2f0-43e0-acd5-1cbe43ea0bb5 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:15:17.581438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-79237df0-f2f0-43e0-acd5-1cbe43ea0bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:17.593702 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79237df0-f2f0-43e0-acd5-1cbe43ea0bb5 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:15:17.597443 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d686e8-76e4-438d-b9da-898c82a0a2d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:15:17.598168 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d686e8-76e4-438d-b9da-898c82a0a2d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:17.598520 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d686e8-76e4-438d-b9da-898c82a0a2d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:17.598866 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d686e8-76e4-438d-b9da-898c82a0a2d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:17.608214 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d686e8-76e4-438d-b9da-898c82a0a2d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:15:17.609719 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d686e8-76e4-438d-b9da-898c82a0a2d6 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:15:17.619378 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1206/2414] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:15:17 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:15:19.167157 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbbeca0-a34c-41fa-a986-dfcefaa82ce5 service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:15:19.178944 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbbeca0-a34c-41fa-a986-dfcefaa82ce5 service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:15:19.184713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02a408-b694-47ec-a1d9-b13bb05ed387 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:15:19.185432 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02a408-b694-47ec-a1d9-b13bb05ed387 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:19.185792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02a408-b694-47ec-a1d9-b13bb05ed387 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:19.186143 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02a408-b694-47ec-a1d9-b13bb05ed387 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:19.188052 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3e02a408-b694-47ec-a1d9-b13bb05ed387 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:19.201464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e02a408-b694-47ec-a1d9-b13bb05ed387 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:15:19.203090 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e02a408-b694-47ec-a1d9-b13bb05ed387 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:15:19.213827 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1209/2415] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:15:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:15:27.615205 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=9 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:15:29.209373 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=12 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:15:46.900014 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb96d0-e3a8-4bf2-9dd5-9a125df54c5e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:46.900488 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb96d0-e3a8-4bf2-9dd5-9a125df54c5e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:46.900624 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb96d0-e3a8-4bf2-9dd5-9a125df54c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:46.919360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-eedb96d0-e3a8-4bf2-9dd5-9a125df54c5e None None] MFA Rules not processed for user `0542cd47ba77454ba25170292ed9aeb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:15:46.920593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-eedb96d0-e3a8-4bf2-9dd5-9a125df54c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:46.976482 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1207/2416] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:15:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:15:47.273823 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02a408-b694-47ec-a1d9-b13bb05ed387 service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:15:47.283192 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02a408-b694-47ec-a1d9-b13bb05ed387 service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:15:47.286820 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df0970-e11a-42b1-81cb-96cdfbaf46bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:15:47.287543 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df0970-e11a-42b1-81cb-96cdfbaf46bb service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:47.287902 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df0970-e11a-42b1-81cb-96cdfbaf46bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:47.288281 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df0970-e11a-42b1-81cb-96cdfbaf46bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:47.290141 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-83df0970-e11a-42b1-81cb-96cdfbaf46bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:47.300736 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df0970-e11a-42b1-81cb-96cdfbaf46bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:15:47.302204 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df0970-e11a-42b1-81cb-96cdfbaf46bb service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:15:47.310948 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1210/2417] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:15:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:15:56.969782 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=25 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:15:57.306772 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:15:58.712948 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89896a9-8484-47db-9915-6cc52487771d None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:58.713506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89896a9-8484-47db-9915-6cc52487771d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:58.713616 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89896a9-8484-47db-9915-6cc52487771d None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:58.715697 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1208/2418] 10.222.0.94 () {54 vars in 921 bytes} [Mon Jul 18 14:15:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:15:58.721532 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803dbe0-d627-4443-8787-531c5ab4883d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:58.722003 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803dbe0-d627-4443-8787-531c5ab4883d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:58.722112 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803dbe0-d627-4443-8787-531c5ab4883d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:58.742861 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1803dbe0-d627-4443-8787-531c5ab4883d None None] MFA Rules not processed for user `14027a0e13fd4272bd7803e3131b6348`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:15:58.744124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1803dbe0-d627-4443-8787-531c5ab4883d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:58.752337 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1211/2419] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Jul 18 14:15:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:15:58.766306 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89896a9-8484-47db-9915-6cc52487771d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:15:58.781827 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89896a9-8484-47db-9915-6cc52487771d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:15:58.786651 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d57713-fed4-4bb3-94a7-0cc20292d20d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:15:58.787329 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d57713-fed4-4bb3-94a7-0cc20292d20d service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:58.787618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d57713-fed4-4bb3-94a7-0cc20292d20d service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:58.787902 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d57713-fed4-4bb3-94a7-0cc20292d20d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:58.796235 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d57713-fed4-4bb3-94a7-0cc20292d20d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:15:58.797730 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d57713-fed4-4bb3-94a7-0cc20292d20d service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:15:58.806716 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1209/2420] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:15:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:15:58.895899 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61897a8-5574-4baa-b4c4-4c9c58016f9d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:58.896222 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61897a8-5574-4baa-b4c4-4c9c58016f9d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:58.896504 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61897a8-5574-4baa-b4c4-4c9c58016f9d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:58.897882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1212/2421] 10.222.0.94 () {54 vars in 921 bytes} [Mon Jul 18 14:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:15:58.905295 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab866f10-2475-4b18-a7e9-5fc6fc8c4206 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:58.905717 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab866f10-2475-4b18-a7e9-5fc6fc8c4206 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:58.906035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab866f10-2475-4b18-a7e9-5fc6fc8c4206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:58.926275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ab866f10-2475-4b18-a7e9-5fc6fc8c4206 None None] MFA Rules not processed for user `ac717fcd9fc1499db12cf852d45572a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:15:58.927493 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ab866f10-2475-4b18-a7e9-5fc6fc8c4206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:58.937597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1210/2422] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Jul 18 14:15:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:15:58.956959 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61897a8-5574-4baa-b4c4-4c9c58016f9d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:15:58.968880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61897a8-5574-4baa-b4c4-4c9c58016f9d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:15:58.973204 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0d6a3-3978-4519-9bc4-f46f5d235155 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:15:58.974065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0d6a3-3978-4519-9bc4-f46f5d235155 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:58.974364 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0d6a3-3978-4519-9bc4-f46f5d235155 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:58.974777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0d6a3-3978-4519-9bc4-f46f5d235155 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:58.986196 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe0d6a3-3978-4519-9bc4-f46f5d235155 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:15:58.987692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe0d6a3-3978-4519-9bc4-f46f5d235155 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:15:59.000196 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1213/2423] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:15:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:16:08.920358 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=9 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:16:08.994528 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=9 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:16:10.466286 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab866f10-2475-4b18-a7e9-5fc6fc8c4206 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:16:10.477500 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab866f10-2475-4b18-a7e9-5fc6fc8c4206 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:16:10.483048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42366e08-67d0-492f-96cd-63f603193d48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:16:10.483692 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42366e08-67d0-492f-96cd-63f603193d48 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:16:10.484006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42366e08-67d0-492f-96cd-63f603193d48 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:16:10.484293 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42366e08-67d0-492f-96cd-63f603193d48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:16:10.486058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-42366e08-67d0-492f-96cd-63f603193d48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:16:10.542609 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42366e08-67d0-492f-96cd-63f603193d48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:16:10.544251 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42366e08-67d0-492f-96cd-63f603193d48 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:16:10.553499 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1211/2424] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:16:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:16:20.549403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=28 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:16:39.739376 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0d6a3-3978-4519-9bc4-f46f5d235155 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:16:39.758601 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0d6a3-3978-4519-9bc4-f46f5d235155 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:16:39.762882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc4e9a-3429-47f3-91a7-97641176f029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:16:39.763532 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc4e9a-3429-47f3-91a7-97641176f029 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:16:39.763820 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc4e9a-3429-47f3-91a7-97641176f029 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:16:39.764125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc4e9a-3429-47f3-91a7-97641176f029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:16:39.765893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8acc4e9a-3429-47f3-91a7-97641176f029 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:16:39.876316 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc4e9a-3429-47f3-91a7-97641176f029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:16:39.877763 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc4e9a-3429-47f3-91a7-97641176f029 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:16:39.896204 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1214/2425] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:16:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:16:43.405715 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42366e08-67d0-492f-96cd-63f603193d48 service nova] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:16:43.407204 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-42366e08-67d0-492f-96cd-63f603193d48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:16:43.455561 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42366e08-67d0-492f-96cd-63f603193d48 service nova] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:16:43.460282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da8f16-52ce-4475-ada5-e161ddb4fb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:16:43.461152 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da8f16-52ce-4475-ada5-e161ddb4fb27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:16:43.461438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da8f16-52ce-4475-ada5-e161ddb4fb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:16:43.461716 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da8f16-52ce-4475-ada5-e161ddb4fb27 admin admin] PATH_INFO: `/v3/users/3720c2225b51464abee34d05a69b9db7` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:16:43.462923 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da8f16-52ce-4475-ada5-e161ddb4fb27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3720c2225b51464abee34d05a69b9db7)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:16:43.465513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da8f16-52ce-4475-ada5-e161ddb4fb27 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:16:43.511810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-96da8f16-52ce-4475-ada5-e161ddb4fb27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3720c2225b51464abee34d05a69b9db7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:16:43.519609 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-96da8f16-52ce-4475-ada5-e161ddb4fb27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3720c2225b51464abee34d05a69b9db7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:16:43.527964 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-96da8f16-52ce-4475-ada5-e161ddb4fb27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3720c2225b51464abee34d05a69b9db7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:16:43.528659 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-96da8f16-52ce-4475-ada5-e161ddb4fb27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3720c2225b51464abee34d05a69b9db7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:16:43.546007 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-96da8f16-52ce-4475-ada5-e161ddb4fb27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3720c2225b51464abee34d05a69b9db7'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:16:43.557887 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1212/2426] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:16:43 2022] DELETE /identity/v3/users/3720c2225b51464abee34d05a69b9db7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:16:43.736610 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc4e9a-3429-47f3-91a7-97641176f029 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:16:43.746782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc4e9a-3429-47f3-91a7-97641176f029 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:16:43.806934 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e16f29-2828-490e-8f3d-81b80d119377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:16:43.807825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e16f29-2828-490e-8f3d-81b80d119377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:16:43.808199 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e16f29-2828-490e-8f3d-81b80d119377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:16:43.808429 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e16f29-2828-490e-8f3d-81b80d119377 admin admin] PATH_INFO: `/v3/projects/eaf9f75abc034491bd1b8846c05bf625` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:16:43.809609 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e16f29-2828-490e-8f3d-81b80d119377 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaf9f75abc034491bd1b8846c05bf625)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:16:43.811502 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e16f29-2828-490e-8f3d-81b80d119377 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:16:43.842283 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:16:43.857722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-16e16f29-2828-490e-8f3d-81b80d119377 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaf9f75abc034491bd1b8846c05bf625', 'request_id': 'req-16e16f29-2828-490e-8f3d-81b80d119377'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:16:43.863683 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-16e16f29-2828-490e-8f3d-81b80d119377 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaf9f75abc034491bd1b8846c05bf625', 'request_id': 'req-16e16f29-2828-490e-8f3d-81b80d119377'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:16:43.871046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-16e16f29-2828-490e-8f3d-81b80d119377 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaf9f75abc034491bd1b8846c05bf625', 'request_id': 'req-16e16f29-2828-490e-8f3d-81b80d119377'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:16:43.881228 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-16e16f29-2828-490e-8f3d-81b80d119377 admin admin] The token cache is being invalidate because project eaf9f75abc034491bd1b8846c05bf625 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:16:43.881518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-16e16f29-2828-490e-8f3d-81b80d119377 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:16:43.882136 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-16e16f29-2828-490e-8f3d-81b80d119377 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:16:43.883645 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1215/2427] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:16:43 2022] DELETE /identity/v3/projects/eaf9f75abc034491bd1b8846c05bf625 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:16:43.889698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26358bcf-2d4f-48d7-92b9-83c91cf36602 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:16:43.889990 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26358bcf-2d4f-48d7-92b9-83c91cf36602 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:16:43.890272 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26358bcf-2d4f-48d7-92b9-83c91cf36602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:16:43.948332 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-26358bcf-2d4f-48d7-92b9-83c91cf36602 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:16:43.950083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-26358bcf-2d4f-48d7-92b9-83c91cf36602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:16:44.010808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1213/2428] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:16:44.019263 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca9e9e-9b73-4347-ab19-96ff8874e9fa None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:16:44.019552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca9e9e-9b73-4347-ab19-96ff8874e9fa None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:16:44.019841 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca9e9e-9b73-4347-ab19-96ff8874e9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:16:44.037601 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6aca9e9e-9b73-4347-ab19-96ff8874e9fa None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:16:44.038804 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6aca9e9e-9b73-4347-ab19-96ff8874e9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:16:44.046611 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1216/2429] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:16:44.050981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26358bcf-2d4f-48d7-92b9-83c91cf36602 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:16:44.061445 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26358bcf-2d4f-48d7-92b9-83c91cf36602 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:16:44.066081 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38533d36-dc7e-4f7c-bc38-182ac2949789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:16:44.066733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38533d36-dc7e-4f7c-bc38-182ac2949789 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:16:44.067026 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38533d36-dc7e-4f7c-bc38-182ac2949789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:16:44.067328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38533d36-dc7e-4f7c-bc38-182ac2949789 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:16:44.068090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38533d36-dc7e-4f7c-bc38-182ac2949789 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:16:44.068403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38533d36-dc7e-4f7c-bc38-182ac2949789 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:16:44.069991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38533d36-dc7e-4f7c-bc38-182ac2949789 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:16:44.072359 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:16:44.087445 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1214/2430] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:16:44.088629 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca9e9e-9b73-4347-ab19-96ff8874e9fa None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:16:44.098029 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca9e9e-9b73-4347-ab19-96ff8874e9fa None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:16:44.102696 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b606b-84a1-48a1-ab43-241c5cdebba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:16:44.103340 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b606b-84a1-48a1-ab43-241c5cdebba6 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:16:44.103624 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b606b-84a1-48a1-ab43-241c5cdebba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:16:44.103898 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b606b-84a1-48a1-ab43-241c5cdebba6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:16:44.104609 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b606b-84a1-48a1-ab43-241c5cdebba6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:16:44.106534 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b606b-84a1-48a1-ab43-241c5cdebba6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:16:44.131151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1217/2431] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:16:44 2022] POST /identity/v3/projects => generated 389 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:16:44.135403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38533d36-dc7e-4f7c-bc38-182ac2949789 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:16:44.147477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38533d36-dc7e-4f7c-bc38-182ac2949789 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:16:44.205756 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e15aa-f495-4d49-a170-9dbdb5655e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:16:44.206435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e15aa-f495-4d49-a170-9dbdb5655e89 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:16:44.206725 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e15aa-f495-4d49-a170-9dbdb5655e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:16:44.207008 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e15aa-f495-4d49-a170-9dbdb5655e89 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:16:44.207698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e15aa-f495-4d49-a170-9dbdb5655e89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:16:44.209776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e15aa-f495-4d49-a170-9dbdb5655e89 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:16:44.229364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1215/2432] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:16:44 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:16:44.233443 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b606b-84a1-48a1-ab43-241c5cdebba6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:16:44.243544 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b606b-84a1-48a1-ab43-241c5cdebba6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:16:44.248830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386c5e6-3852-4e70-a665-899e2a91587d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:16:44.249486 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386c5e6-3852-4e70-a665-899e2a91587d admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:16:44.249778 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386c5e6-3852-4e70-a665-899e2a91587d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:16:44.250062 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386c5e6-3852-4e70-a665-899e2a91587d admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:16:44.250786 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386c5e6-3852-4e70-a665-899e2a91587d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:16:44.252438 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386c5e6-3852-4e70-a665-899e2a91587d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:16:44.263597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1218/2433] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:16:44.267569 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e15aa-f495-4d49-a170-9dbdb5655e89 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:16:44.277515 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e15aa-f495-4d49-a170-9dbdb5655e89 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:16:44.282122 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b2b8ae-e2e7-422f-b2d5-1324ff6fdb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:16:44.282774 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2b8ae-e2e7-422f-b2d5-1324ff6fdb48 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:16:44.283063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2b8ae-e2e7-422f-b2d5-1324ff6fdb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:16:44.283345 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2b8ae-e2e7-422f-b2d5-1324ff6fdb48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:16:44.284124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b2b8ae-e2e7-422f-b2d5-1324ff6fdb48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:16:44.285705 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b2b8ae-e2e7-422f-b2d5-1324ff6fdb48 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:16:44.296068 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1216/2434] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:16:44.300284 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386c5e6-3852-4e70-a665-899e2a91587d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:16:44.310920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386c5e6-3852-4e70-a665-899e2a91587d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:16:44.315626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e657b-249d-4ec7-80ad-fb0d33444edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:16:44.316473 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e657b-249d-4ec7-80ad-fb0d33444edf admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:16:44.316764 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e657b-249d-4ec7-80ad-fb0d33444edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:16:44.317046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e657b-249d-4ec7-80ad-fb0d33444edf admin admin] PATH_INFO: `/v3/projects/3e0348cc781c4af3bfc3302499b19c74/users/8561df96cd7c47c89d283e36fc637bb0/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:16:44.331291 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e657b-249d-4ec7-80ad-fb0d33444edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e0348cc781c4af3bfc3302499b19c74, user_id=8561df96cd7c47c89d283e36fc637bb0, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:16:44.333998 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e657b-249d-4ec7-80ad-fb0d33444edf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:16:44.348597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1219/2435] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:16:44 2022] PUT /identity/v3/projects/3e0348cc781c4af3bfc3302499b19c74/users/8561df96cd7c47c89d283e36fc637bb0/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:16:44.353780 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2316882-0dac-44e8-a849-10ba4c0d26ea None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:16:44.354082 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2316882-0dac-44e8-a849-10ba4c0d26ea None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:16:44.354368 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2316882-0dac-44e8-a849-10ba4c0d26ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:16:44.373038 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-e2316882-0dac-44e8-a849-10ba4c0d26ea None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:16:44.374237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e2316882-0dac-44e8-a849-10ba4c0d26ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:16:44.431606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1217/2436] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:16:44.443798 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e657b-249d-4ec7-80ad-fb0d33444edf admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:16:44.445532 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6b3e657b-249d-4ec7-80ad-fb0d33444edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:16:44.531771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e657b-249d-4ec7-80ad-fb0d33444edf admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:16:44.535380 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68828df2-4eb6-43ff-a3eb-6aaf91d18bf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:16:44.536169 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68828df2-4eb6-43ff-a3eb-6aaf91d18bf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:16:44.536456 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68828df2-4eb6-43ff-a3eb-6aaf91d18bf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:16:44.536731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68828df2-4eb6-43ff-a3eb-6aaf91d18bf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:16:44.546112 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68828df2-4eb6-43ff-a3eb-6aaf91d18bf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:16:44.547791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68828df2-4eb6-43ff-a3eb-6aaf91d18bf9 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:16:44.557073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1220/2437] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:16:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:16:48.447151 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ad221-a353-47cf-82ed-4c4bda055552 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:16:48.447617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ad221-a353-47cf-82ed-4c4bda055552 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:16:48.447827 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ad221-a353-47cf-82ed-4c4bda055552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:16:48.484991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c15ad221-a353-47cf-82ed-4c4bda055552 None None] MFA Rules not processed for user `8561df96cd7c47c89d283e36fc637bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:16:48.486238 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c15ad221-a353-47cf-82ed-4c4bda055552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:16:48.555152 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1218/2438] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:16:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:16:48.570475 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68828df2-4eb6-43ff-a3eb-6aaf91d18bf9 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:16:48.571869 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-68828df2-4eb6-43ff-a3eb-6aaf91d18bf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:16:48.603975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:16:48.696771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68828df2-4eb6-43ff-a3eb-6aaf91d18bf9 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:16:48.702422 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1650c94-4c4e-4a24-b18e-dd9dfeedd32c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:16:48.703074 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1650c94-4c4e-4a24-b18e-dd9dfeedd32c service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:16:48.703368 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1650c94-4c4e-4a24-b18e-dd9dfeedd32c service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:16:48.703663 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1650c94-4c4e-4a24-b18e-dd9dfeedd32c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:16:48.711691 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1650c94-4c4e-4a24-b18e-dd9dfeedd32c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:16:48.714076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1650c94-4c4e-4a24-b18e-dd9dfeedd32c service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:16:48.726869 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1221/2439] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:16:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:16:50.052765 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591164c5-72ba-4136-aff1-46e9a370b228 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:16:50.053413 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591164c5-72ba-4136-aff1-46e9a370b228 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:16:50.053413 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591164c5-72ba-4136-aff1-46e9a370b228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:16:50.060689 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:16:50.072371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-591164c5-72ba-4136-aff1-46e9a370b228 None None] MFA Rules not processed for user `8561df96cd7c47c89d283e36fc637bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:16:50.074180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-591164c5-72ba-4136-aff1-46e9a370b228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:16:50.081585 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1219/2440] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:16:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:16:51.188330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1650c94-4c4e-4a24-b18e-dd9dfeedd32c service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:16:51.189883 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b1650c94-4c4e-4a24-b18e-dd9dfeedd32c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:16:51.311356 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1650c94-4c4e-4a24-b18e-dd9dfeedd32c service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:16:51.317446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c72b0f-9257-435f-9613-3370161cdf7e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd5b2ba9e37041c78a99f1700a4970c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:16:51.318297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c72b0f-9257-435f-9613-3370161cdf7e service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:16:51.318593 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c72b0f-9257-435f-9613-3370161cdf7e service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:16:51.318924 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c72b0f-9257-435f-9613-3370161cdf7e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:16:51.320822 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-07c72b0f-9257-435f-9613-3370161cdf7e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:16:51.334060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c72b0f-9257-435f-9613-3370161cdf7e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:16:51.335618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c72b0f-9257-435f-9613-3370161cdf7e service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:16:51.347488 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1222/2441] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:16:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:17:00.069661 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:17:01.342626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=75 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:17:04.223061 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e61cb6-da1c-4b47-98c6-84321b683edc None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:04.223532 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e61cb6-da1c-4b47-98c6-84321b683edc None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:04.223630 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e61cb6-da1c-4b47-98c6-84321b683edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:04.241411 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d2e61cb6-da1c-4b47-98c6-84321b683edc None None] MFA Rules not processed for user `0b6cb92018ea4cc5a25c5423b9bcd7a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:17:04.242608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d2e61cb6-da1c-4b47-98c6-84321b683edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:17:04.296680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1220/2442] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:17:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:17:14.289216 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=25 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:17:42.599314 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1675d2-2097-475e-868a-3543b5126a00 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:42.599842 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1675d2-2097-475e-868a-3543b5126a00 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:42.599842 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1675d2-2097-475e-868a-3543b5126a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:42.616564 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ab1675d2-2097-475e-868a-3543b5126a00 None None] MFA Rules not processed for user `0542cd47ba77454ba25170292ed9aeb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:17:42.617765 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ab1675d2-2097-475e-868a-3543b5126a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:17:42.672900 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1223/2443] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:17:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:17:52.665937 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=25 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:17:54.680636 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8cb7c-5e04-43cc-a471-ca580cbfd87e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:54.681213 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8cb7c-5e04-43cc-a471-ca580cbfd87e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:54.681347 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8cb7c-5e04-43cc-a471-ca580cbfd87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:54.699539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c5a8cb7c-5e04-43cc-a471-ca580cbfd87e None None] MFA Rules not processed for user `35334a0f92254b5e900aad8c51873809`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:17:54.700768 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c5a8cb7c-5e04-43cc-a471-ca580cbfd87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:17:54.756824 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1221/2444] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:17:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:18:04.415589 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1675d2-2097-475e-868a-3543b5126a00 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:04.417074 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ab1675d2-2097-475e-868a-3543b5126a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:04.429433 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1675d2-2097-475e-868a-3543b5126a00 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:04.433700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920dc13-6607-4389-bdb8-8b5ef4a12b55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:04.434344 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920dc13-6607-4389-bdb8-8b5ef4a12b55 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:04.434635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920dc13-6607-4389-bdb8-8b5ef4a12b55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:04.434947 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920dc13-6607-4389-bdb8-8b5ef4a12b55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:04.436838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9920dc13-6607-4389-bdb8-8b5ef4a12b55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:04.448618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920dc13-6607-4389-bdb8-8b5ef4a12b55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:04.450054 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920dc13-6607-4389-bdb8-8b5ef4a12b55 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:04.459577 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1224/2445] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:18:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:18:04.749762 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=25 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:18:07.410824 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a8cb7c-5e04-43cc-a471-ca580cbfd87e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:07.424837 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a8cb7c-5e04-43cc-a471-ca580cbfd87e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:07.430888 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23826fd6-3039-46d3-83f9-31df292db405 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd5b2ba9e37041c78a99f1700a4970c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:07.431570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23826fd6-3039-46d3-83f9-31df292db405 service placement] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:07.431856 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23826fd6-3039-46d3-83f9-31df292db405 service placement] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:07.432227 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23826fd6-3039-46d3-83f9-31df292db405 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:07.434175 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-23826fd6-3039-46d3-83f9-31df292db405 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:07.449009 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23826fd6-3039-46d3-83f9-31df292db405 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:07.450640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23826fd6-3039-46d3-83f9-31df292db405 service placement] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:07.462666 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1222/2446] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:18:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:18:10.576250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920dc13-6607-4389-bdb8-8b5ef4a12b55 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:10.578720 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9920dc13-6607-4389-bdb8-8b5ef4a12b55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:10.603220 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920dc13-6607-4389-bdb8-8b5ef4a12b55 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:10.607880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b8606-028d-45d4-a4f6-c52d2d95892e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:10.608798 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b8606-028d-45d4-a4f6-c52d2d95892e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:10.609083 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b8606-028d-45d4-a4f6-c52d2d95892e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:10.609356 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b8606-028d-45d4-a4f6-c52d2d95892e admin admin] PATH_INFO: `/v3/users/2aa4925080114a33b80e2ca588fbeb27` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:10.610551 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b8606-028d-45d4-a4f6-c52d2d95892e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aa4925080114a33b80e2ca588fbeb27)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:10.612535 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b8606-028d-45d4-a4f6-c52d2d95892e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:10.657411 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-453b8606-028d-45d4-a4f6-c52d2d95892e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aa4925080114a33b80e2ca588fbeb27'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:18:10.665142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-453b8606-028d-45d4-a4f6-c52d2d95892e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aa4925080114a33b80e2ca588fbeb27'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:18:10.672520 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-453b8606-028d-45d4-a4f6-c52d2d95892e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aa4925080114a33b80e2ca588fbeb27'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:18:10.673153 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-453b8606-028d-45d4-a4f6-c52d2d95892e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aa4925080114a33b80e2ca588fbeb27'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:18:10.685819 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-453b8606-028d-45d4-a4f6-c52d2d95892e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aa4925080114a33b80e2ca588fbeb27'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:18:10.691607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1225/2447] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:18:10 2022] DELETE /identity/v3/users/2aa4925080114a33b80e2ca588fbeb27 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:18:10.883579 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23826fd6-3039-46d3-83f9-31df292db405 service placement] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:10.900860 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23826fd6-3039-46d3-83f9-31df292db405 service placement] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:10.958441 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325ae35-08e4-4bce-a82e-e0d644515808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:10.959311 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325ae35-08e4-4bce-a82e-e0d644515808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:10.959603 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325ae35-08e4-4bce-a82e-e0d644515808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:10.959886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325ae35-08e4-4bce-a82e-e0d644515808 admin admin] PATH_INFO: `/v3/projects/f618e7dff5d7411eb4d4cade50d68607` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:10.961690 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325ae35-08e4-4bce-a82e-e0d644515808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f618e7dff5d7411eb4d4cade50d68607)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:10.963634 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325ae35-08e4-4bce-a82e-e0d644515808 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:11.008641 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c325ae35-08e4-4bce-a82e-e0d644515808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f618e7dff5d7411eb4d4cade50d68607', 'request_id': 'req-c325ae35-08e4-4bce-a82e-e0d644515808'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:18:11.014072 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c325ae35-08e4-4bce-a82e-e0d644515808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f618e7dff5d7411eb4d4cade50d68607', 'request_id': 'req-c325ae35-08e4-4bce-a82e-e0d644515808'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:18:11.020881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c325ae35-08e4-4bce-a82e-e0d644515808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f618e7dff5d7411eb4d4cade50d68607', 'request_id': 'req-c325ae35-08e4-4bce-a82e-e0d644515808'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:18:11.032680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c325ae35-08e4-4bce-a82e-e0d644515808 admin admin] The token cache is being invalidate because project f618e7dff5d7411eb4d4cade50d68607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:18:11.032972 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c325ae35-08e4-4bce-a82e-e0d644515808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:18:11.033604 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-c325ae35-08e4-4bce-a82e-e0d644515808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:18:11.035127 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1223/2448] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:18:10 2022] DELETE /identity/v3/projects/f618e7dff5d7411eb4d4cade50d68607 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:18:11.039088 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b8606-028d-45d4-a4f6-c52d2d95892e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:11.041177 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-453b8606-028d-45d4-a4f6-c52d2d95892e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:11.054238 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b8606-028d-45d4-a4f6-c52d2d95892e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:11.109261 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98ede1-8a16-459b-ad3e-918e7e66df90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:11.110129 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98ede1-8a16-459b-ad3e-918e7e66df90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:11.110419 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98ede1-8a16-459b-ad3e-918e7e66df90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:11.110702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98ede1-8a16-459b-ad3e-918e7e66df90 admin admin] PATH_INFO: `/v3/users/b7d5e46b2dcb4246a8267eb3654b9e74` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:11.111887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98ede1-8a16-459b-ad3e-918e7e66df90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7d5e46b2dcb4246a8267eb3654b9e74)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:11.113845 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98ede1-8a16-459b-ad3e-918e7e66df90 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:11.158742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ac98ede1-8a16-459b-ad3e-918e7e66df90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7d5e46b2dcb4246a8267eb3654b9e74'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:18:11.165708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ac98ede1-8a16-459b-ad3e-918e7e66df90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7d5e46b2dcb4246a8267eb3654b9e74'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:18:11.171463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:18:11.173631 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ac98ede1-8a16-459b-ad3e-918e7e66df90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7d5e46b2dcb4246a8267eb3654b9e74'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:18:11.174537 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ac98ede1-8a16-459b-ad3e-918e7e66df90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7d5e46b2dcb4246a8267eb3654b9e74'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:18:11.187390 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ac98ede1-8a16-459b-ad3e-918e7e66df90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7d5e46b2dcb4246a8267eb3654b9e74'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:18:11.193486 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1226/2449] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:18:11 2022] DELETE /identity/v3/users/b7d5e46b2dcb4246a8267eb3654b9e74 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:18:11.359244 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325ae35-08e4-4bce-a82e-e0d644515808 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:11.360718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c325ae35-08e4-4bce-a82e-e0d644515808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:11.375097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325ae35-08e4-4bce-a82e-e0d644515808 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:11.434418 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a17076-5456-44f7-965d-3bbd99edc69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:11.435202 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a17076-5456-44f7-965d-3bbd99edc69e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:11.435485 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a17076-5456-44f7-965d-3bbd99edc69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:11.435763 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a17076-5456-44f7-965d-3bbd99edc69e admin admin] PATH_INFO: `/v3/projects/d8b0bbd989be47ea916196e8f8b4d8f1` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:11.436955 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a17076-5456-44f7-965d-3bbd99edc69e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8b0bbd989be47ea916196e8f8b4d8f1)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:11.438859 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a17076-5456-44f7-965d-3bbd99edc69e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:11.461076 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:18:11.489523 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-b6a17076-5456-44f7-965d-3bbd99edc69e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8b0bbd989be47ea916196e8f8b4d8f1', 'request_id': 'req-b6a17076-5456-44f7-965d-3bbd99edc69e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:18:11.494898 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-b6a17076-5456-44f7-965d-3bbd99edc69e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8b0bbd989be47ea916196e8f8b4d8f1', 'request_id': 'req-b6a17076-5456-44f7-965d-3bbd99edc69e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:18:11.502038 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-b6a17076-5456-44f7-965d-3bbd99edc69e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8b0bbd989be47ea916196e8f8b4d8f1', 'request_id': 'req-b6a17076-5456-44f7-965d-3bbd99edc69e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:18:11.514415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-b6a17076-5456-44f7-965d-3bbd99edc69e admin admin] The token cache is being invalidate because project d8b0bbd989be47ea916196e8f8b4d8f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:18:11.514715 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-b6a17076-5456-44f7-965d-3bbd99edc69e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:18:11.515379 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-b6a17076-5456-44f7-965d-3bbd99edc69e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:18:11.517178 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1224/2450] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:18:11 2022] DELETE /identity/v3/projects/d8b0bbd989be47ea916196e8f8b4d8f1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:18:21.189792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:18:21.511267 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:18:34.519630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98ede1-8a16-459b-ad3e-918e7e66df90 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:34.521056 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ac98ede1-8a16-459b-ad3e-918e7e66df90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:34.576357 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98ede1-8a16-459b-ad3e-918e7e66df90 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:34.624228 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b814d-4fac-4237-9043-a5c13a004302 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '503a8228d4754c309fc3f365f4cfc61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:34.624881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b814d-4fac-4237-9043-a5c13a004302 service glance] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:34.625170 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b814d-4fac-4237-9043-a5c13a004302 service glance] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:34.625449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b814d-4fac-4237-9043-a5c13a004302 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:34.628212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-db1b814d-4fac-4237-9043-a5c13a004302 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:34.696877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b814d-4fac-4237-9043-a5c13a004302 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:34.698243 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b814d-4fac-4237-9043-a5c13a004302 service glance] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:34.709544 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1227/2451] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:18:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:18:34.954559 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a17076-5456-44f7-965d-3bbd99edc69e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:34.956053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b6a17076-5456-44f7-965d-3bbd99edc69e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:35.015071 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a17076-5456-44f7-965d-3bbd99edc69e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:35.018839 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676bd71f-f9a4-41f0-b3b8-a673333b35a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:35.019687 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bd71f-f9a4-41f0-b3b8-a673333b35a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:35.020001 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bd71f-f9a4-41f0-b3b8-a673333b35a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:35.020305 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bd71f-f9a4-41f0-b3b8-a673333b35a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:35.022075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-676bd71f-f9a4-41f0-b3b8-a673333b35a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:35.035615 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676bd71f-f9a4-41f0-b3b8-a673333b35a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:35.037083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676bd71f-f9a4-41f0-b3b8-a673333b35a8 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:35.048286 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1225/2452] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:18:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:18:43.949245 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b814d-4fac-4237-9043-a5c13a004302 service glance] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:43.950668 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-db1b814d-4fac-4237-9043-a5c13a004302 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:44.013860 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:18:44.028689 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b814d-4fac-4237-9043-a5c13a004302 service glance] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:44.032471 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4c966-d4fd-4779-ac8f-13e53a8df7e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:44.033125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4c966-d4fd-4779-ac8f-13e53a8df7e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:44.033417 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4c966-d4fd-4779-ac8f-13e53a8df7e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:44.033701 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4c966-d4fd-4779-ac8f-13e53a8df7e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:44.035516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-01e4c966-d4fd-4779-ac8f-13e53a8df7e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:44.113219 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4c966-d4fd-4779-ac8f-13e53a8df7e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:44.114661 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4c966-d4fd-4779-ac8f-13e53a8df7e0 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:44.133834 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1228/2453] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:18:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:18:45.043442 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=26 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:18:54.130586 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=44 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:01.004322 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61909e38-ad41-48fe-9241-0fe8e5e27b01 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:01.004829 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61909e38-ad41-48fe-9241-0fe8e5e27b01 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:01.004932 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61909e38-ad41-48fe-9241-0fe8e5e27b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:01.071566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-61909e38-ad41-48fe-9241-0fe8e5e27b01 None None] MFA Rules not processed for user `94d824bffc2c4af299c537056bc21734`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:01.073004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-61909e38-ad41-48fe-9241-0fe8e5e27b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:01.154848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1226/2454] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Jul 18 14:19:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:01.202672 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4c966-d4fd-4779-ac8f-13e53a8df7e0 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:01.204178 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-01e4c966-d4fd-4779-ac8f-13e53a8df7e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:01.311145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4c966-d4fd-4779-ac8f-13e53a8df7e0 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:01.316389 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf29ea5-b2bd-446c-9add-831d56b0b5a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbf513ee25374e49b3d8ce28843e393a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:01.317041 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf29ea5-b2bd-446c-9add-831d56b0b5a6 service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:01.317334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf29ea5-b2bd-446c-9add-831d56b0b5a6 service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:01.317618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf29ea5-b2bd-446c-9add-831d56b0b5a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:01.326128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf29ea5-b2bd-446c-9add-831d56b0b5a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:01.327741 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf29ea5-b2bd-446c-9add-831d56b0b5a6 service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:01.337681 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1229/2455] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Jul 18 14:19:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:19:08.063253 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61909e38-ad41-48fe-9241-0fe8e5e27b01 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:08.074525 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61909e38-ad41-48fe-9241-0fe8e5e27b01 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:08.079468 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6cba25-e0ef-41f8-a43f-5aa27e34f9d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:08.080382 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cba25-e0ef-41f8-a43f-5aa27e34f9d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:08.081182 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cba25-e0ef-41f8-a43f-5aa27e34f9d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:08.081945 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cba25-e0ef-41f8-a43f-5aa27e34f9d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:08.084229 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-4c6cba25-e0ef-41f8-a43f-5aa27e34f9d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:08.157749 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6cba25-e0ef-41f8-a43f-5aa27e34f9d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:08.159612 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6cba25-e0ef-41f8-a43f-5aa27e34f9d6 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:08.173714 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1227/2456] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:19:11.333262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=41 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:11.951219 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf29ea5-b2bd-446c-9add-831d56b0b5a6 service swift] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:11.953470 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dbf29ea5-b2bd-446c-9add-831d56b0b5a6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:11.970893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf29ea5-b2bd-446c-9add-831d56b0b5a6 service swift] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:11.975540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd11aca-49da-4399-ad35-7d1b037a9410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:11.976553 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd11aca-49da-4399-ad35-7d1b037a9410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:11.976838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd11aca-49da-4399-ad35-7d1b037a9410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:11.977188 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd11aca-49da-4399-ad35-7d1b037a9410 admin admin] PATH_INFO: `/v3/users/0b6cb92018ea4cc5a25c5423b9bcd7a9` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:11.978485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd11aca-49da-4399-ad35-7d1b037a9410 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b6cb92018ea4cc5a25c5423b9bcd7a9)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:11.980591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd11aca-49da-4399-ad35-7d1b037a9410 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:12.027584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-dbd11aca-49da-4399-ad35-7d1b037a9410 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b6cb92018ea4cc5a25c5423b9bcd7a9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:12.034555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-dbd11aca-49da-4399-ad35-7d1b037a9410 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b6cb92018ea4cc5a25c5423b9bcd7a9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:12.042068 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-dbd11aca-49da-4399-ad35-7d1b037a9410 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b6cb92018ea4cc5a25c5423b9bcd7a9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:12.042712 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-dbd11aca-49da-4399-ad35-7d1b037a9410 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b6cb92018ea4cc5a25c5423b9bcd7a9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:12.054338 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-dbd11aca-49da-4399-ad35-7d1b037a9410 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b6cb92018ea4cc5a25c5423b9bcd7a9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:12.060217 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1230/2457] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:19:11 2022] DELETE /identity/v3/users/0b6cb92018ea4cc5a25c5423b9bcd7a9 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:12.112065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6cba25-e0ef-41f8-a43f-5aa27e34f9d6 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:12.120863 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6cba25-e0ef-41f8-a43f-5aa27e34f9d6 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:12.171359 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77a71d-53c3-4902-a3ed-2f2f081fcf02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:12.172095 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77a71d-53c3-4902-a3ed-2f2f081fcf02 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:12.172380 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77a71d-53c3-4902-a3ed-2f2f081fcf02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:12.172680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77a71d-53c3-4902-a3ed-2f2f081fcf02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:12.174446 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-dc77a71d-53c3-4902-a3ed-2f2f081fcf02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:12.183634 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77a71d-53c3-4902-a3ed-2f2f081fcf02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:12.185085 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc77a71d-53c3-4902-a3ed-2f2f081fcf02 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:12.189534 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:12.196252 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1228/2458] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:12.447814 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd11aca-49da-4399-ad35-7d1b037a9410 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:12.450472 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dbd11aca-49da-4399-ad35-7d1b037a9410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:12.521713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd11aca-49da-4399-ad35-7d1b037a9410 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:12.527330 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3219fa2-0710-413d-a8b0-d7ec8818f97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:12.527731 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3219fa2-0710-413d-a8b0-d7ec8818f97f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:12.528083 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3219fa2-0710-413d-a8b0-d7ec8818f97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:12.528360 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3219fa2-0710-413d-a8b0-d7ec8818f97f admin admin] PATH_INFO: `/v3/projects/31755010e81b43908bbef9ae75eaaa64` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:12.529549 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3219fa2-0710-413d-a8b0-d7ec8818f97f admin admin] RBAC: Authorizing `identity:delete_project(project_id=31755010e81b43908bbef9ae75eaaa64)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:12.531461 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3219fa2-0710-413d-a8b0-d7ec8818f97f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:12.585845 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f3219fa2-0710-413d-a8b0-d7ec8818f97f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31755010e81b43908bbef9ae75eaaa64', 'request_id': 'req-f3219fa2-0710-413d-a8b0-d7ec8818f97f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:12.591935 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f3219fa2-0710-413d-a8b0-d7ec8818f97f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31755010e81b43908bbef9ae75eaaa64', 'request_id': 'req-f3219fa2-0710-413d-a8b0-d7ec8818f97f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:12.600214 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f3219fa2-0710-413d-a8b0-d7ec8818f97f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31755010e81b43908bbef9ae75eaaa64', 'request_id': 'req-f3219fa2-0710-413d-a8b0-d7ec8818f97f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:12.611175 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f3219fa2-0710-413d-a8b0-d7ec8818f97f admin admin] The token cache is being invalidate because project 31755010e81b43908bbef9ae75eaaa64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:19:12.611466 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f3219fa2-0710-413d-a8b0-d7ec8818f97f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:12.612123 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f3219fa2-0710-413d-a8b0-d7ec8818f97f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:12.614230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1231/2459] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:19:12 2022] DELETE /identity/v3/projects/31755010e81b43908bbef9ae75eaaa64 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:12.621868 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd244b-dd64-469b-8917-ba722060324f None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:12.622233 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd244b-dd64-469b-8917-ba722060324f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:12.622602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd244b-dd64-469b-8917-ba722060324f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:12.640769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-aabd244b-dd64-469b-8917-ba722060324f None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:12.641962 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-aabd244b-dd64-469b-8917-ba722060324f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:12.702980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1229/2460] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:12.711074 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32941d91-2d72-4804-829e-94c394daf643 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:12.711560 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32941d91-2d72-4804-829e-94c394daf643 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:12.711956 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32941d91-2d72-4804-829e-94c394daf643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:12.718490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:12.734896 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-32941d91-2d72-4804-829e-94c394daf643 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:12.736242 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-32941d91-2d72-4804-829e-94c394daf643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:12.745154 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1232/2461] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:12.749521 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd244b-dd64-469b-8917-ba722060324f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:12.759417 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd244b-dd64-469b-8917-ba722060324f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:12.764247 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a985e-7ca3-4ce2-a4be-6fe0d1d10119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:12.764914 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a985e-7ca3-4ce2-a4be-6fe0d1d10119 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:12.765232 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a985e-7ca3-4ce2-a4be-6fe0d1d10119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:12.765489 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a985e-7ca3-4ce2-a4be-6fe0d1d10119 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:12.766264 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a985e-7ca3-4ce2-a4be-6fe0d1d10119 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:19:12.766575 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a985e-7ca3-4ce2-a4be-6fe0d1d10119 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:12.768316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a985e-7ca3-4ce2-a4be-6fe0d1d10119 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:12.780232 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1230/2462] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:19:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:19:12.784569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32941d91-2d72-4804-829e-94c394daf643 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:12.794707 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32941d91-2d72-4804-829e-94c394daf643 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:12.799326 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cc4a8-a3e4-4b3a-8e0f-997e2d467480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:12.800007 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cc4a8-a3e4-4b3a-8e0f-997e2d467480 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:12.800645 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cc4a8-a3e4-4b3a-8e0f-997e2d467480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:12.800645 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cc4a8-a3e4-4b3a-8e0f-997e2d467480 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:12.801238 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7cc4a8-a3e4-4b3a-8e0f-997e2d467480 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:12.803265 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7cc4a8-a3e4-4b3a-8e0f-997e2d467480 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:12.838828 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1233/2463] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:19:12 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:12.843351 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a985e-7ca3-4ce2-a4be-6fe0d1d10119 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:12.853835 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a985e-7ca3-4ce2-a4be-6fe0d1d10119 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:12.907954 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc2aae-3c7f-477c-bb41-64711e640c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:12.908618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc2aae-3c7f-477c-bb41-64711e640c39 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:12.908904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc2aae-3c7f-477c-bb41-64711e640c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:12.909180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc2aae-3c7f-477c-bb41-64711e640c39 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:12.909872 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc2aae-3c7f-477c-bb41-64711e640c39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:12.911774 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc2aae-3c7f-477c-bb41-64711e640c39 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:12.932190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1231/2464] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:19:12 2022] POST /identity/v3/users => generated 293 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:12.936271 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cc4a8-a3e4-4b3a-8e0f-997e2d467480 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:12.947349 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cc4a8-a3e4-4b3a-8e0f-997e2d467480 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:12.952116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be54feb-a39b-4e19-bb55-6d544f9341fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:12.952777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be54feb-a39b-4e19-bb55-6d544f9341fc admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:12.953069 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be54feb-a39b-4e19-bb55-6d544f9341fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:12.953356 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be54feb-a39b-4e19-bb55-6d544f9341fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:12.954087 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be54feb-a39b-4e19-bb55-6d544f9341fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:12.955699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be54feb-a39b-4e19-bb55-6d544f9341fc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:12.965717 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1234/2465] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:12.969741 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc2aae-3c7f-477c-bb41-64711e640c39 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:12.980364 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc2aae-3c7f-477c-bb41-64711e640c39 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:12.985036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c152985-5f5d-4e21-b3e1-9324a2f914e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:12.985695 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c152985-5f5d-4e21-b3e1-9324a2f914e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:12.985988 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c152985-5f5d-4e21-b3e1-9324a2f914e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:12.986273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c152985-5f5d-4e21-b3e1-9324a2f914e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:12.987005 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c152985-5f5d-4e21-b3e1-9324a2f914e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:12.988661 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c152985-5f5d-4e21-b3e1-9324a2f914e0 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:12.999127 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1232/2466] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:13.003197 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be54feb-a39b-4e19-bb55-6d544f9341fc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:13.013116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be54feb-a39b-4e19-bb55-6d544f9341fc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:13.017781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acab693-5714-4911-bbcf-0cd05b59280b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:13.018659 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acab693-5714-4911-bbcf-0cd05b59280b admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:13.018946 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acab693-5714-4911-bbcf-0cd05b59280b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:13.019242 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acab693-5714-4911-bbcf-0cd05b59280b admin admin] PATH_INFO: `/v3/projects/ff18e668769c415385b3d7a2b13ae454/users/f1bebe5708a14d6289b518d5330d8775/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:13.033489 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acab693-5714-4911-bbcf-0cd05b59280b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff18e668769c415385b3d7a2b13ae454, user_id=f1bebe5708a14d6289b518d5330d8775, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:13.036196 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acab693-5714-4911-bbcf-0cd05b59280b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:13.049312 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1235/2467] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:13 2022] PUT /identity/v3/projects/ff18e668769c415385b3d7a2b13ae454/users/f1bebe5708a14d6289b518d5330d8775/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:13.059694 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb118a98-d39f-4aba-ac32-1c82d56e6cdb None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:13.060009 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb118a98-d39f-4aba-ac32-1c82d56e6cdb None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:13.060302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb118a98-d39f-4aba-ac32-1c82d56e6cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:13.086609 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-fb118a98-d39f-4aba-ac32-1c82d56e6cdb None None] MFA Rules not processed for user `f1bebe5708a14d6289b518d5330d8775`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:13.087844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fb118a98-d39f-4aba-ac32-1c82d56e6cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:13.142079 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:13.155069 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1233/2468] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:19:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:13.156452 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acab693-5714-4911-bbcf-0cd05b59280b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:13.217666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acab693-5714-4911-bbcf-0cd05b59280b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:13.222871 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754de42d-e72f-44bb-97dd-64d9cc1528ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:13.224397 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754de42d-e72f-44bb-97dd-64d9cc1528ba admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:13.224850 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754de42d-e72f-44bb-97dd-64d9cc1528ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:13.225144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754de42d-e72f-44bb-97dd-64d9cc1528ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:13.226457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754de42d-e72f-44bb-97dd-64d9cc1528ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:13.228395 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754de42d-e72f-44bb-97dd-64d9cc1528ba admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:13.253891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1236/2469] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:19:13 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:13.257837 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb118a98-d39f-4aba-ac32-1c82d56e6cdb None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:13.268553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb118a98-d39f-4aba-ac32-1c82d56e6cdb None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:13.323699 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e465a1f-20cc-413b-b1b6-5f43fe33b4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:13.324396 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e465a1f-20cc-413b-b1b6-5f43fe33b4fd admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:13.324683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e465a1f-20cc-413b-b1b6-5f43fe33b4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:13.324958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e465a1f-20cc-413b-b1b6-5f43fe33b4fd admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:13.325646 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e465a1f-20cc-413b-b1b6-5f43fe33b4fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:13.327555 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e465a1f-20cc-413b-b1b6-5f43fe33b4fd admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:13.347264 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1234/2470] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:19:13 2022] POST /identity/v3/users => generated 299 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:19:13.351204 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754de42d-e72f-44bb-97dd-64d9cc1528ba admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:13.362166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754de42d-e72f-44bb-97dd-64d9cc1528ba admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:13.366928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f928d8-0522-4c14-9f2b-78f074280f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:13.367586 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f928d8-0522-4c14-9f2b-78f074280f76 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:13.367879 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f928d8-0522-4c14-9f2b-78f074280f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:13.368199 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f928d8-0522-4c14-9f2b-78f074280f76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:13.368925 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f928d8-0522-4c14-9f2b-78f074280f76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:13.370526 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f928d8-0522-4c14-9f2b-78f074280f76 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:13.380970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1237/2471] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:13.385031 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e465a1f-20cc-413b-b1b6-5f43fe33b4fd admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:13.394895 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e465a1f-20cc-413b-b1b6-5f43fe33b4fd admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:13.399905 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62b414-9092-4706-b7b0-020a758e73b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:13.400810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62b414-9092-4706-b7b0-020a758e73b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:13.401100 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62b414-9092-4706-b7b0-020a758e73b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:13.401383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62b414-9092-4706-b7b0-020a758e73b8 admin admin] PATH_INFO: `/v3/projects/e9469eca094f4215b1b81e580cc4eb1e/users/eac91229aecf4745b566b306e270cf26/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:13.416696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62b414-9092-4706-b7b0-020a758e73b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9469eca094f4215b1b81e580cc4eb1e, user_id=eac91229aecf4745b566b306e270cf26, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:13.419581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62b414-9092-4706-b7b0-020a758e73b8 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:13.432673 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1235/2472] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:13 2022] PUT /identity/v3/projects/e9469eca094f4215b1b81e580cc4eb1e/users/eac91229aecf4745b566b306e270cf26/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:13.436672 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f928d8-0522-4c14-9f2b-78f074280f76 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:13.447329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f928d8-0522-4c14-9f2b-78f074280f76 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:13.502338 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc5650-80fa-4edf-adb1-e5aa71486cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:13.503063 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc5650-80fa-4edf-adb1-e5aa71486cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:13.503354 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc5650-80fa-4edf-adb1-e5aa71486cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:13.503633 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc5650-80fa-4edf-adb1-e5aa71486cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:13.504377 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc5650-80fa-4edf-adb1-e5aa71486cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:13.505974 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc5650-80fa-4edf-adb1-e5aa71486cf0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:13.511494 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:13.517172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1238/2473] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:13.521172 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62b414-9092-4706-b7b0-020a758e73b8 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:13.532548 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62b414-9092-4706-b7b0-020a758e73b8 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:13.537228 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688dc215-7f84-4992-9e2a-b24808f7ccaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:13.538146 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dc215-7f84-4992-9e2a-b24808f7ccaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:13.538435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dc215-7f84-4992-9e2a-b24808f7ccaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:13.538718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dc215-7f84-4992-9e2a-b24808f7ccaa admin admin] PATH_INFO: `/v3/projects/e9469eca094f4215b1b81e580cc4eb1e/users/eac91229aecf4745b566b306e270cf26/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:13.541237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688dc215-7f84-4992-9e2a-b24808f7ccaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9469eca094f4215b1b81e580cc4eb1e, user_id=eac91229aecf4745b566b306e270cf26, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:13.544008 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688dc215-7f84-4992-9e2a-b24808f7ccaa admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:13.554401 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1236/2474] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:13 2022] PUT /identity/v3/projects/e9469eca094f4215b1b81e580cc4eb1e/users/eac91229aecf4745b566b306e270cf26/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:13.565954 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f968b1-3d60-4a82-bfea-f089464bfbbe None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:13.566281 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f968b1-3d60-4a82-bfea-f089464bfbbe None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:13.566568 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f968b1-3d60-4a82-bfea-f089464bfbbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:13.608258 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b5f968b1-3d60-4a82-bfea-f089464bfbbe None None] MFA Rules not processed for user `eac91229aecf4745b566b306e270cf26`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:13.609743 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b5f968b1-3d60-4a82-bfea-f089464bfbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:13.679188 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1239/2475] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:19:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:13.689747 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688dc215-7f84-4992-9e2a-b24808f7ccaa admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:13.691762 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-688dc215-7f84-4992-9e2a-b24808f7ccaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:13.754272 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688dc215-7f84-4992-9e2a-b24808f7ccaa admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:13.758538 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de94559-2a77-45fd-96a2-a7bcbbcf8e35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:13.759195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de94559-2a77-45fd-96a2-a7bcbbcf8e35 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:13.759486 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de94559-2a77-45fd-96a2-a7bcbbcf8e35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:13.759766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de94559-2a77-45fd-96a2-a7bcbbcf8e35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:13.770230 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de94559-2a77-45fd-96a2-a7bcbbcf8e35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:13.771719 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de94559-2a77-45fd-96a2-a7bcbbcf8e35 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:13.781924 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1237/2476] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:13 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:14.312788 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f968b1-3d60-4a82-bfea-f089464bfbbe None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:14.373542 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f968b1-3d60-4a82-bfea-f089464bfbbe None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:14.378006 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e0a4f-ec4c-4d58-8701-2d569fa28715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:14.378889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e0a4f-ec4c-4d58-8701-2d569fa28715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:14.379204 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e0a4f-ec4c-4d58-8701-2d569fa28715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:14.379484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e0a4f-ec4c-4d58-8701-2d569fa28715 admin admin] PATH_INFO: `/v3/users/f1bebe5708a14d6289b518d5330d8775` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:14.380692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e0a4f-ec4c-4d58-8701-2d569fa28715 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1bebe5708a14d6289b518d5330d8775)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:14.382703 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e0a4f-ec4c-4d58-8701-2d569fa28715 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:14.427192 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-aa5e0a4f-ec4c-4d58-8701-2d569fa28715 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1bebe5708a14d6289b518d5330d8775'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:14.436025 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-aa5e0a4f-ec4c-4d58-8701-2d569fa28715 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1bebe5708a14d6289b518d5330d8775'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:14.447243 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-aa5e0a4f-ec4c-4d58-8701-2d569fa28715 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1bebe5708a14d6289b518d5330d8775'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:14.448579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-aa5e0a4f-ec4c-4d58-8701-2d569fa28715 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1bebe5708a14d6289b518d5330d8775'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:14.461074 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-aa5e0a4f-ec4c-4d58-8701-2d569fa28715 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1bebe5708a14d6289b518d5330d8775'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:14.467163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1240/2477] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:19:14 2022] DELETE /identity/v3/users/f1bebe5708a14d6289b518d5330d8775 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:14.471598 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8f077-4252-48c8-97b7-c300c5f23a20 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:14.472091 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8f077-4252-48c8-97b7-c300c5f23a20 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:14.472240 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8f077-4252-48c8-97b7-c300c5f23a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:14.493246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-adc8f077-4252-48c8-97b7-c300c5f23a20 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:14.494499 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-adc8f077-4252-48c8-97b7-c300c5f23a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:14.541697 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:14.558136 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1238/2478] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:19:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:14.567918 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e0a4f-ec4c-4d58-8701-2d569fa28715 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:14.569403 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-aa5e0a4f-ec4c-4d58-8701-2d569fa28715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:14.576934 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:14.631083 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e0a4f-ec4c-4d58-8701-2d569fa28715 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:14.635403 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d89ed-e717-4b66-b819-1993af24aaa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:14.636091 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d89ed-e717-4b66-b819-1993af24aaa8 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:14.636381 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d89ed-e717-4b66-b819-1993af24aaa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:14.636664 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d89ed-e717-4b66-b819-1993af24aaa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:14.645953 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d89ed-e717-4b66-b819-1993af24aaa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:14.648667 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d89ed-e717-4b66-b819-1993af24aaa8 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:14.658094 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1241/2479] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:14.953760 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8f077-4252-48c8-97b7-c300c5f23a20 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:14.955391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-adc8f077-4252-48c8-97b7-c300c5f23a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:14.968839 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8f077-4252-48c8-97b7-c300c5f23a20 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:14.973885 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ea551-340b-4e27-9524-04bcc1510ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:14.974878 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ea551-340b-4e27-9524-04bcc1510ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:14.975317 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ea551-340b-4e27-9524-04bcc1510ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:14.975708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ea551-340b-4e27-9524-04bcc1510ec4 admin admin] PATH_INFO: `/v3/projects/ff18e668769c415385b3d7a2b13ae454` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:14.977049 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ea551-340b-4e27-9524-04bcc1510ec4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff18e668769c415385b3d7a2b13ae454)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:14.979037 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ea551-340b-4e27-9524-04bcc1510ec4 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:15.030259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-549ea551-340b-4e27-9524-04bcc1510ec4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff18e668769c415385b3d7a2b13ae454', 'request_id': 'req-549ea551-340b-4e27-9524-04bcc1510ec4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:15.036636 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-549ea551-340b-4e27-9524-04bcc1510ec4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff18e668769c415385b3d7a2b13ae454', 'request_id': 'req-549ea551-340b-4e27-9524-04bcc1510ec4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:15.044790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-549ea551-340b-4e27-9524-04bcc1510ec4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff18e668769c415385b3d7a2b13ae454', 'request_id': 'req-549ea551-340b-4e27-9524-04bcc1510ec4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:15.056631 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-549ea551-340b-4e27-9524-04bcc1510ec4 admin admin] The token cache is being invalidate because project ff18e668769c415385b3d7a2b13ae454 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:19:15.056925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-549ea551-340b-4e27-9524-04bcc1510ec4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:15.057600 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-549ea551-340b-4e27-9524-04bcc1510ec4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:15.059161 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1239/2480] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:19:14 2022] DELETE /identity/v3/projects/ff18e668769c415385b3d7a2b13ae454 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:15.063010 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d89ed-e717-4b66-b819-1993af24aaa8 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:15.073686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d89ed-e717-4b66-b819-1993af24aaa8 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:15.134646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf6f7a9-f134-4079-94f3-e7d59e78bd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:15.135506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6f7a9-f134-4079-94f3-e7d59e78bd55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:15.136083 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6f7a9-f134-4079-94f3-e7d59e78bd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:15.136083 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6f7a9-f134-4079-94f3-e7d59e78bd55 admin admin] PATH_INFO: `/v3/users/eac91229aecf4745b566b306e270cf26` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:15.137342 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf6f7a9-f134-4079-94f3-e7d59e78bd55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eac91229aecf4745b566b306e270cf26)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:15.139283 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf6f7a9-f134-4079-94f3-e7d59e78bd55 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:15.185666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5cf6f7a9-f134-4079-94f3-e7d59e78bd55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eac91229aecf4745b566b306e270cf26'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:15.193231 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5cf6f7a9-f134-4079-94f3-e7d59e78bd55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eac91229aecf4745b566b306e270cf26'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:15.200918 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5cf6f7a9-f134-4079-94f3-e7d59e78bd55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eac91229aecf4745b566b306e270cf26'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:15.201573 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5cf6f7a9-f134-4079-94f3-e7d59e78bd55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eac91229aecf4745b566b306e270cf26'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:15.214907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5cf6f7a9-f134-4079-94f3-e7d59e78bd55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eac91229aecf4745b566b306e270cf26'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:15.221781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1242/2481] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:19:15 2022] DELETE /identity/v3/users/eac91229aecf4745b566b306e270cf26 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:15.523423 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ea551-340b-4e27-9524-04bcc1510ec4 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:15.525504 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-549ea551-340b-4e27-9524-04bcc1510ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:15.526167 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65aa4a1-f641-4d84-899e-f84bdf516ba4 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:15.526560 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65aa4a1-f641-4d84-899e-f84bdf516ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:15.527034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65aa4a1-f641-4d84-899e-f84bdf516ba4 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:15.529222 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1243/2482] 10.222.0.94 () {54 vars in 921 bytes} [Mon Jul 18 14:19:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:19:15.536708 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cd414-5abf-435a-ad55-777907b244fe None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:15.537246 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cd414-5abf-435a-ad55-777907b244fe None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:15.537637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cd414-5abf-435a-ad55-777907b244fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:15.562138 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ea551-340b-4e27-9524-04bcc1510ec4 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:15.599604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0e1cd414-5abf-435a-ad55-777907b244fe None None] MFA Rules not processed for user `ac717fcd9fc1499db12cf852d45572a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:15.600935 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0e1cd414-5abf-435a-ad55-777907b244fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:15.611091 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:15.639638 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf6311-a55a-49ca-b3ff-452395e853ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:15.640803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf6311-a55a-49ca-b3ff-452395e853ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:15.641195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf6311-a55a-49ca-b3ff-452395e853ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:15.641580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf6311-a55a-49ca-b3ff-452395e853ae admin admin] PATH_INFO: `/v3/projects/e9469eca094f4215b1b81e580cc4eb1e` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:15.643024 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf6311-a55a-49ca-b3ff-452395e853ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9469eca094f4215b1b81e580cc4eb1e)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:15.645057 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf6311-a55a-49ca-b3ff-452395e853ae admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:15.691688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1244/2483] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Jul 18 14:19:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:19:15.700996 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5ccf6311-a55a-49ca-b3ff-452395e853ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9469eca094f4215b1b81e580cc4eb1e', 'request_id': 'req-5ccf6311-a55a-49ca-b3ff-452395e853ae'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:15.703870 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1cd414-5abf-435a-ad55-777907b244fe None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:15.705442 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0e1cd414-5abf-435a-ad55-777907b244fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:15.706492 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5ccf6311-a55a-49ca-b3ff-452395e853ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9469eca094f4215b1b81e580cc4eb1e', 'request_id': 'req-5ccf6311-a55a-49ca-b3ff-452395e853ae'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:15.713926 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5ccf6311-a55a-49ca-b3ff-452395e853ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9469eca094f4215b1b81e580cc4eb1e', 'request_id': 'req-5ccf6311-a55a-49ca-b3ff-452395e853ae'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:15.725495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5ccf6311-a55a-49ca-b3ff-452395e853ae admin admin] The token cache is being invalidate because project e9469eca094f4215b1b81e580cc4eb1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:19:15.725803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5ccf6311-a55a-49ca-b3ff-452395e853ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:15.726448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5ccf6311-a55a-49ca-b3ff-452395e853ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:15.728004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1240/2484] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:19:15 2022] DELETE /identity/v3/projects/e9469eca094f4215b1b81e580cc4eb1e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:15.732674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230659d-94d3-40a6-b79e-f8b8807cad13 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:15.732963 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230659d-94d3-40a6-b79e-f8b8807cad13 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:15.733241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230659d-94d3-40a6-b79e-f8b8807cad13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:15.740106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:15.752724 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d230659d-94d3-40a6-b79e-f8b8807cad13 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:15.754214 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d230659d-94d3-40a6-b79e-f8b8807cad13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:15.762136 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1cd414-5abf-435a-ad55-777907b244fe None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:15.812652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc01ce9-c4be-4fa8-8445-1d28d2ec126b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '503a8228d4754c309fc3f365f4cfc61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:15.813222 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc01ce9-c4be-4fa8-8445-1d28d2ec126b service glance] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:15.813509 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc01ce9-c4be-4fa8-8445-1d28d2ec126b service glance] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:15.813791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc01ce9-c4be-4fa8-8445-1d28d2ec126b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:15.815560 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4fc01ce9-c4be-4fa8-8445-1d28d2ec126b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:15.817544 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1241/2485] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:19:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:19:15.825203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282909c-47df-429d-9669-619ee3c394f5 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:15.825497 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282909c-47df-429d-9669-619ee3c394f5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:15.825780 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282909c-47df-429d-9669-619ee3c394f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:15.843285 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-f282909c-47df-429d-9669-619ee3c394f5 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:15.844932 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f282909c-47df-429d-9669-619ee3c394f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:15.853505 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1242/2486] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:19:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:19:15.857619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282909c-47df-429d-9669-619ee3c394f5 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:15.867184 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282909c-47df-429d-9669-619ee3c394f5 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:15.871985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ea337-8726-4030-9358-203de2249845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:15.872658 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ea337-8726-4030-9358-203de2249845 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:15.872951 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ea337-8726-4030-9358-203de2249845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:15.873231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ea337-8726-4030-9358-203de2249845 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:15.873958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ea337-8726-4030-9358-203de2249845 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:19:15.874272 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ea337-8726-4030-9358-203de2249845 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:15.875904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ea337-8726-4030-9358-203de2249845 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:15.890764 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1243/2487] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:19:15.890931 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc01ce9-c4be-4fa8-8445-1d28d2ec126b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:15.890931 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc01ce9-c4be-4fa8-8445-1d28d2ec126b service glance] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:15.891687 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ea337-8726-4030-9358-203de2249845 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:15.900199 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1245/2488] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:19:15.901175 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ea337-8726-4030-9358-203de2249845 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:15.905672 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875a1db-b088-4a11-bf4b-7b77b33e2f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:15.906955 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875a1db-b088-4a11-bf4b-7b77b33e2f82 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:15.907993 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875a1db-b088-4a11-bf4b-7b77b33e2f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:15.907993 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875a1db-b088-4a11-bf4b-7b77b33e2f82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:15.908265 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875a1db-b088-4a11-bf4b-7b77b33e2f82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:15.910181 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875a1db-b088-4a11-bf4b-7b77b33e2f82 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:15.937897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1244/2489] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:19:15 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:15.942034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc01ce9-c4be-4fa8-8445-1d28d2ec126b service glance] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:15.949289 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:15.956334 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc01ce9-c4be-4fa8-8445-1d28d2ec126b service glance] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:15.978209 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eaa5eb-a8c8-4f71-a7ab-08fd5e85dd31 None None] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:15.978513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eaa5eb-a8c8-4f71-a7ab-08fd5e85dd31 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:15.978802 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eaa5eb-a8c8-4f71-a7ab-08fd5e85dd31 None None] PATH_INFO: `/` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:15.980218 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1245/2490] 10.222.0.94 () {54 vars in 921 bytes} [Mon Jul 18 14:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:19:15.985126 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341e125-5cc9-4b15-bb07-b1b7f5e14cd1 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:15.985537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341e125-5cc9-4b15-bb07-b1b7f5e14cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:15.986593 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341e125-5cc9-4b15-bb07-b1b7f5e14cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:16.017124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ecec8f-7f2f-478e-a548-36f731791896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:16.017377 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecec8f-7f2f-478e-a548-36f731791896 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:16.017662 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecec8f-7f2f-478e-a548-36f731791896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:16.017938 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ecec8f-7f2f-478e-a548-36f731791896 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:16.018637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ecec8f-7f2f-478e-a548-36f731791896 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:16.020584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ecec8f-7f2f-478e-a548-36f731791896 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:16.025153 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-a341e125-5cc9-4b15-bb07-b1b7f5e14cd1 None None] MFA Rules not processed for user `14027a0e13fd4272bd7803e3131b6348`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:16.026473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a341e125-5cc9-4b15-bb07-b1b7f5e14cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:16.041369 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1246/2491] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:19:15 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:19:16.045246 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ecec8f-7f2f-478e-a548-36f731791896 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:16.054564 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ecec8f-7f2f-478e-a548-36f731791896 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:16.059087 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336c694-a8e5-48e4-9609-31cada02919f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:16.059717 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336c694-a8e5-48e4-9609-31cada02919f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:16.060034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336c694-a8e5-48e4-9609-31cada02919f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:16.060310 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336c694-a8e5-48e4-9609-31cada02919f admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:16.061020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336c694-a8e5-48e4-9609-31cada02919f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:16.062577 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336c694-a8e5-48e4-9609-31cada02919f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:16.072877 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1247/2492] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:19:16.076803 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336c694-a8e5-48e4-9609-31cada02919f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:16.081625 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1246/2493] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Jul 18 14:19:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:19:16.086262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336c694-a8e5-48e4-9609-31cada02919f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:16.091053 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd499f-8cf8-48e6-a3ee-e43231809058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:16.091701 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd499f-8cf8-48e6-a3ee-e43231809058 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:16.092014 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd499f-8cf8-48e6-a3ee-e43231809058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:16.092279 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd499f-8cf8-48e6-a3ee-e43231809058 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:16.092998 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd499f-8cf8-48e6-a3ee-e43231809058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:16.093508 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341e125-5cc9-4b15-bb07-b1b7f5e14cd1 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:16.094609 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd499f-8cf8-48e6-a3ee-e43231809058 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:16.094850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a341e125-5cc9-4b15-bb07-b1b7f5e14cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:16.105021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341e125-5cc9-4b15-bb07-b1b7f5e14cd1 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:16.108377 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1248/2494] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:19:16.108910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da888d-3f05-4ad1-a55e-b22e7e125916 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:16.109308 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd499f-8cf8-48e6-a3ee-e43231809058 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:16.109548 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da888d-3f05-4ad1-a55e-b22e7e125916 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:16.109831 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da888d-3f05-4ad1-a55e-b22e7e125916 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:16.110107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da888d-3f05-4ad1-a55e-b22e7e125916 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:16.118304 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da888d-3f05-4ad1-a55e-b22e7e125916 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:16.119080 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd499f-8cf8-48e6-a3ee-e43231809058 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:16.119897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da888d-3f05-4ad1-a55e-b22e7e125916 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:16.123654 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41dc00-f3c9-4ffa-83a9-72aed60c8b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:16.124525 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41dc00-f3c9-4ffa-83a9-72aed60c8b15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:16.124806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41dc00-f3c9-4ffa-83a9-72aed60c8b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:16.125083 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41dc00-f3c9-4ffa-83a9-72aed60c8b15 admin admin] PATH_INFO: `/v3/projects/26309529c7324e148b469b5427ffe288/users/59a26ea9021c4f2ba6f2ba02ecd0f433/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:16.128187 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1247/2495] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:19:16.140542 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41dc00-f3c9-4ffa-83a9-72aed60c8b15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26309529c7324e148b469b5427ffe288, user_id=59a26ea9021c4f2ba6f2ba02ecd0f433, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:16.143206 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41dc00-f3c9-4ffa-83a9-72aed60c8b15 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:16.156783 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1249/2496] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:16 2022] PUT /identity/v3/projects/26309529c7324e148b469b5427ffe288/users/59a26ea9021c4f2ba6f2ba02ecd0f433/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:16.167436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fbd76-29da-4fc1-9451-a077dfae1d34 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:16.167733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fbd76-29da-4fc1-9451-a077dfae1d34 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:16.168050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fbd76-29da-4fc1-9451-a077dfae1d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:16.196377 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-580fbd76-29da-4fc1-9451-a077dfae1d34 None None] MFA Rules not processed for user `59a26ea9021c4f2ba6f2ba02ecd0f433`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:16.197682 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-580fbd76-29da-4fc1-9451-a077dfae1d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:16.211987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffac4e-1a82-4268-897e-ad6b06f4c685 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:16.212356 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffac4e-1a82-4268-897e-ad6b06f4c685 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:16.212638 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffac4e-1a82-4268-897e-ad6b06f4c685 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:16.214001 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1250/2497] 10.222.0.94 () {54 vars in 921 bytes} [Mon Jul 18 14:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:19:16.219100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630d47f-c867-466e-a542-52a7997acd77 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:16.219467 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630d47f-c867-466e-a542-52a7997acd77 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:16.219758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630d47f-c867-466e-a542-52a7997acd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:16.232950 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:16.238217 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c630d47f-c867-466e-a542-52a7997acd77 None None] MFA Rules not processed for user `ac717fcd9fc1499db12cf852d45572a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:16.239726 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c630d47f-c867-466e-a542-52a7997acd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:16.266793 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1248/2498] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:19:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:16.278321 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fbd76-29da-4fc1-9451-a077dfae1d34 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:16.279698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-580fbd76-29da-4fc1-9451-a077dfae1d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:16.309163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1251/2499] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Jul 18 14:19:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:19:16.319767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630d47f-c867-466e-a542-52a7997acd77 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:16.337347 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fbd76-29da-4fc1-9451-a077dfae1d34 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:16.341128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67784808-f4f7-4807-a5e7-6c22aebc50a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:16.341779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67784808-f4f7-4807-a5e7-6c22aebc50a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:16.342068 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67784808-f4f7-4807-a5e7-6c22aebc50a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:16.342349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67784808-f4f7-4807-a5e7-6c22aebc50a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:16.351312 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67784808-f4f7-4807-a5e7-6c22aebc50a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:16.352776 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67784808-f4f7-4807-a5e7-6c22aebc50a8 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:16.360541 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:16.361689 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1249/2500] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:19:16.375067 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630d47f-c867-466e-a542-52a7997acd77 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:16.378862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233aa73-4f14-49ab-a8c8-46b56c355ca2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:16.379658 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233aa73-4f14-49ab-a8c8-46b56c355ca2 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:16.380250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233aa73-4f14-49ab-a8c8-46b56c355ca2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:16.380546 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233aa73-4f14-49ab-a8c8-46b56c355ca2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:16.381554 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67784808-f4f7-4807-a5e7-6c22aebc50a8 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:16.391619 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d233aa73-4f14-49ab-a8c8-46b56c355ca2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:16.393125 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d233aa73-4f14-49ab-a8c8-46b56c355ca2 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:16.404351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1252/2501] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:19:16.443509 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67784808-f4f7-4807-a5e7-6c22aebc50a8 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:16.448659 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c79103-576c-48e6-9c3b-2b331c6da19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:16.449535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c79103-576c-48e6-9c3b-2b331c6da19a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:16.449821 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c79103-576c-48e6-9c3b-2b331c6da19a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:16.450100 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c79103-576c-48e6-9c3b-2b331c6da19a admin admin] PATH_INFO: `/v3/users/59a26ea9021c4f2ba6f2ba02ecd0f433` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:16.451301 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c79103-576c-48e6-9c3b-2b331c6da19a admin admin] RBAC: Authorizing `identity:delete_user(user_id=59a26ea9021c4f2ba6f2ba02ecd0f433)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:16.453264 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c79103-576c-48e6-9c3b-2b331c6da19a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:16.502032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-15c79103-576c-48e6-9c3b-2b331c6da19a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59a26ea9021c4f2ba6f2ba02ecd0f433'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:16.510669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-15c79103-576c-48e6-9c3b-2b331c6da19a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59a26ea9021c4f2ba6f2ba02ecd0f433'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:16.517892 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-15c79103-576c-48e6-9c3b-2b331c6da19a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59a26ea9021c4f2ba6f2ba02ecd0f433'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:16.518561 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-15c79103-576c-48e6-9c3b-2b331c6da19a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59a26ea9021c4f2ba6f2ba02ecd0f433'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:16.531422 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-15c79103-576c-48e6-9c3b-2b331c6da19a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59a26ea9021c4f2ba6f2ba02ecd0f433'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:16.532918 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:16.538856 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1250/2502] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:19:16 2022] DELETE /identity/v3/users/59a26ea9021c4f2ba6f2ba02ecd0f433 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:19:16.544227 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46dd928-b699-418c-b964-478cd1dab813 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:16.544605 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46dd928-b699-418c-b964-478cd1dab813 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:16.545019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46dd928-b699-418c-b964-478cd1dab813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:16.563116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d46dd928-b699-418c-b964-478cd1dab813 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:16.564374 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d46dd928-b699-418c-b964-478cd1dab813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:16.625345 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1253/2503] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:19:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:16.635444 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c79103-576c-48e6-9c3b-2b331c6da19a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:16.636927 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-15c79103-576c-48e6-9c3b-2b331c6da19a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:16.696674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c79103-576c-48e6-9c3b-2b331c6da19a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:16.700288 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0504a-b62a-440f-bd53-25170ac17ee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:16.700961 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0504a-b62a-440f-bd53-25170ac17ee5 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:16.701247 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0504a-b62a-440f-bd53-25170ac17ee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:16.701530 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0504a-b62a-440f-bd53-25170ac17ee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:16.712302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0504a-b62a-440f-bd53-25170ac17ee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:16.713744 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0504a-b62a-440f-bd53-25170ac17ee5 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:16.725102 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1251/2504] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:17.103399 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46dd928-b699-418c-b964-478cd1dab813 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:17.104935 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d46dd928-b699-418c-b964-478cd1dab813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:17.117040 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46dd928-b699-418c-b964-478cd1dab813 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:17.121513 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbba234-914d-4be6-a16f-b691b5bbae18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:17.122368 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbba234-914d-4be6-a16f-b691b5bbae18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:17.122652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbba234-914d-4be6-a16f-b691b5bbae18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:17.122931 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbba234-914d-4be6-a16f-b691b5bbae18 admin admin] PATH_INFO: `/v3/projects/26309529c7324e148b469b5427ffe288` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:17.124137 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbba234-914d-4be6-a16f-b691b5bbae18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26309529c7324e148b469b5427ffe288)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:17.126101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbba234-914d-4be6-a16f-b691b5bbae18 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:17.174287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9fbba234-914d-4be6-a16f-b691b5bbae18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26309529c7324e148b469b5427ffe288', 'request_id': 'req-9fbba234-914d-4be6-a16f-b691b5bbae18'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:17.180642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9fbba234-914d-4be6-a16f-b691b5bbae18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26309529c7324e148b469b5427ffe288', 'request_id': 'req-9fbba234-914d-4be6-a16f-b691b5bbae18'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:17.188423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9fbba234-914d-4be6-a16f-b691b5bbae18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26309529c7324e148b469b5427ffe288', 'request_id': 'req-9fbba234-914d-4be6-a16f-b691b5bbae18'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:17.199307 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9fbba234-914d-4be6-a16f-b691b5bbae18 admin admin] The token cache is being invalidate because project 26309529c7324e148b469b5427ffe288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:19:17.199591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9fbba234-914d-4be6-a16f-b691b5bbae18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:17.200239 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9fbba234-914d-4be6-a16f-b691b5bbae18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:17.201777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1254/2505] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:19:17 2022] DELETE /identity/v3/projects/26309529c7324e148b469b5427ffe288 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:17.208946 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f64b3-e071-4ca5-9acd-7fc7e287e7eb None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:17.209252 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f64b3-e071-4ca5-9acd-7fc7e287e7eb None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:17.210096 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f64b3-e071-4ca5-9acd-7fc7e287e7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:17.228513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-bb8f64b3-e071-4ca5-9acd-7fc7e287e7eb None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:17.229726 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bb8f64b3-e071-4ca5-9acd-7fc7e287e7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:17.292365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1252/2506] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:17.301082 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29724049-469b-4545-83f6-94ce9c38f375 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:17.301382 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29724049-469b-4545-83f6-94ce9c38f375 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:17.301667 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29724049-469b-4545-83f6-94ce9c38f375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:17.320363 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-29724049-469b-4545-83f6-94ce9c38f375 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:17.321579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-29724049-469b-4545-83f6-94ce9c38f375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:17.329555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1255/2507] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:17.333592 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f64b3-e071-4ca5-9acd-7fc7e287e7eb None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:17.343935 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f64b3-e071-4ca5-9acd-7fc7e287e7eb None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:17.348513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430d1a6-25dc-4cd8-9074-35c3c5c2b9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:17.349165 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430d1a6-25dc-4cd8-9074-35c3c5c2b9b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:17.349453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430d1a6-25dc-4cd8-9074-35c3c5c2b9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:17.349732 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430d1a6-25dc-4cd8-9074-35c3c5c2b9b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:17.350453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4430d1a6-25dc-4cd8-9074-35c3c5c2b9b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:19:17.350762 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4430d1a6-25dc-4cd8-9074-35c3c5c2b9b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:17.352394 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4430d1a6-25dc-4cd8-9074-35c3c5c2b9b7 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:17.364650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1253/2508] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:19:17.368890 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29724049-469b-4545-83f6-94ce9c38f375 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:17.378184 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29724049-469b-4545-83f6-94ce9c38f375 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:17.383333 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed25014-6e6b-42d2-87d5-7412934c62d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:17.384042 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed25014-6e6b-42d2-87d5-7412934c62d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:17.384427 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed25014-6e6b-42d2-87d5-7412934c62d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:17.384717 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed25014-6e6b-42d2-87d5-7412934c62d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:17.385426 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed25014-6e6b-42d2-87d5-7412934c62d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:17.387383 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed25014-6e6b-42d2-87d5-7412934c62d9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:17.413483 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1256/2509] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:19:17 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:19:17.418190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430d1a6-25dc-4cd8-9074-35c3c5c2b9b7 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:17.428733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430d1a6-25dc-4cd8-9074-35c3c5c2b9b7 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:17.487889 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b3caf-f129-4646-bab9-29367acbd237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:17.488591 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b3caf-f129-4646-bab9-29367acbd237 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:17.488882 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b3caf-f129-4646-bab9-29367acbd237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:17.489166 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b3caf-f129-4646-bab9-29367acbd237 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:17.489859 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b3caf-f129-4646-bab9-29367acbd237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:17.492080 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b3caf-f129-4646-bab9-29367acbd237 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:17.523118 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1254/2510] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:19:17 2022] POST /identity/v3/users => generated 285 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:17.527340 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed25014-6e6b-42d2-87d5-7412934c62d9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:17.537971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed25014-6e6b-42d2-87d5-7412934c62d9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:17.543426 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8d946-e056-4e32-9f4c-8258321177be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:17.544153 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8d946-e056-4e32-9f4c-8258321177be admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:17.544463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8d946-e056-4e32-9f4c-8258321177be admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:17.544745 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8d946-e056-4e32-9f4c-8258321177be admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:17.545487 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8d946-e056-4e32-9f4c-8258321177be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:17.547107 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8d946-e056-4e32-9f4c-8258321177be admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:17.560008 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1257/2511] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:17.562709 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b3caf-f129-4646-bab9-29367acbd237 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:17.573347 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b3caf-f129-4646-bab9-29367acbd237 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:17.578320 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d799555-010f-4683-9202-643a646d1311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:17.578984 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d799555-010f-4683-9202-643a646d1311 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:17.579272 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d799555-010f-4683-9202-643a646d1311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:17.579553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d799555-010f-4683-9202-643a646d1311 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:17.580334 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d799555-010f-4683-9202-643a646d1311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:17.581900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d799555-010f-4683-9202-643a646d1311 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:17.592908 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1255/2512] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:19:17.597090 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8d946-e056-4e32-9f4c-8258321177be admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:17.607205 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8d946-e056-4e32-9f4c-8258321177be admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:17.611854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8447b5-059e-4b58-8e55-57a0d0e6db9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:17.612765 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8447b5-059e-4b58-8e55-57a0d0e6db9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:17.613058 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8447b5-059e-4b58-8e55-57a0d0e6db9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:17.613337 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8447b5-059e-4b58-8e55-57a0d0e6db9f admin admin] PATH_INFO: `/v3/projects/768200d8040e4cc4bf9c42c9aab5b73b/users/836c0e30f6ad4e289c90d2dcf0b60202/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:17.630620 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8447b5-059e-4b58-8e55-57a0d0e6db9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=768200d8040e4cc4bf9c42c9aab5b73b, user_id=836c0e30f6ad4e289c90d2dcf0b60202, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:17.634215 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8447b5-059e-4b58-8e55-57a0d0e6db9f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:17.645103 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:17.650160 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1258/2513] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:17 2022] PUT /identity/v3/projects/768200d8040e4cc4bf9c42c9aab5b73b/users/836c0e30f6ad4e289c90d2dcf0b60202/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:17.660513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0277f-cde2-43d4-84a0-af8ec806f596 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:17.660813 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0277f-cde2-43d4-84a0-af8ec806f596 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:17.661101 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0277f-cde2-43d4-84a0-af8ec806f596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:17.673727 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:17.688762 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-01a0277f-cde2-43d4-84a0-af8ec806f596 None None] MFA Rules not processed for user `836c0e30f6ad4e289c90d2dcf0b60202`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:17.689976 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-01a0277f-cde2-43d4-84a0-af8ec806f596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:17.758348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1256/2514] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:19:17 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:17.773549 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8447b5-059e-4b58-8e55-57a0d0e6db9f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:17.775539 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7e8447b5-059e-4b58-8e55-57a0d0e6db9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:17.838552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8447b5-059e-4b58-8e55-57a0d0e6db9f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:17.842527 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14c2e2-fb35-49af-a3c9-79ac8eb92752 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '503a8228d4754c309fc3f365f4cfc61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:17.843180 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14c2e2-fb35-49af-a3c9-79ac8eb92752 service glance] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:17.843468 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14c2e2-fb35-49af-a3c9-79ac8eb92752 service glance] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:17.843748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14c2e2-fb35-49af-a3c9-79ac8eb92752 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:17.852372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14c2e2-fb35-49af-a3c9-79ac8eb92752 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:17.853814 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14c2e2-fb35-49af-a3c9-79ac8eb92752 service glance] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:17.862630 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1259/2515] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:17 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:18.957202 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a631ef-b876-4192-aaca-12c3641374b4 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:18.957667 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a631ef-b876-4192-aaca-12c3641374b4 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:18.957760 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a631ef-b876-4192-aaca-12c3641374b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:18.976459 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c0a631ef-b876-4192-aaca-12c3641374b4 None None] MFA Rules not processed for user `94d824bffc2c4af299c537056bc21734`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:18.978499 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c0a631ef-b876-4192-aaca-12c3641374b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:19.036723 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1257/2516] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Jul 18 14:19:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:19.048806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14c2e2-fb35-49af-a3c9-79ac8eb92752 service glance] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:19.050202 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bd14c2e2-fb35-49af-a3c9-79ac8eb92752 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:19.120140 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14c2e2-fb35-49af-a3c9-79ac8eb92752 service glance] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:19.125497 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f993f-ebc3-46aa-bfa3-134d0f57088b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbf513ee25374e49b3d8ce28843e393a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:19.126148 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f993f-ebc3-46aa-bfa3-134d0f57088b service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:19.126439 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f993f-ebc3-46aa-bfa3-134d0f57088b service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:19.126722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f993f-ebc3-46aa-bfa3-134d0f57088b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:19.136732 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f993f-ebc3-46aa-bfa3-134d0f57088b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:19.138520 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f993f-ebc3-46aa-bfa3-134d0f57088b service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:19.146977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1260/2517] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Jul 18 14:19:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:19:25.448382 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a631ef-b876-4192-aaca-12c3641374b4 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:25.449949 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c0a631ef-b876-4192-aaca-12c3641374b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:25.561304 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a631ef-b876-4192-aaca-12c3641374b4 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:25.567975 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969108e-09b0-4d4a-823a-c69304e1340d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:25.568665 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969108e-09b0-4d4a-823a-c69304e1340d service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:25.569020 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969108e-09b0-4d4a-823a-c69304e1340d service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:25.569306 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969108e-09b0-4d4a-823a-c69304e1340d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:25.571147 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c969108e-09b0-4d4a-823a-c69304e1340d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:25.612651 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:25.660846 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969108e-09b0-4d4a-823a-c69304e1340d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:25.663025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969108e-09b0-4d4a-823a-c69304e1340d service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:25.701294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1258/2518] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:29.143429 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=58 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:35.696728 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=23 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:57.110250 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f993f-ebc3-46aa-bfa3-134d0f57088b service swift] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:57.173929 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f993f-ebc3-46aa-bfa3-134d0f57088b service swift] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:57.178477 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820440e2-163a-4f1e-a97a-d1fb1dafc528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:57.179344 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820440e2-163a-4f1e-a97a-d1fb1dafc528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:57.179627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820440e2-163a-4f1e-a97a-d1fb1dafc528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:57.180031 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820440e2-163a-4f1e-a97a-d1fb1dafc528 admin admin] PATH_INFO: `/v3/users/836c0e30f6ad4e289c90d2dcf0b60202` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:57.181117 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820440e2-163a-4f1e-a97a-d1fb1dafc528 admin admin] RBAC: Authorizing `identity:delete_user(user_id=836c0e30f6ad4e289c90d2dcf0b60202)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:57.183109 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820440e2-163a-4f1e-a97a-d1fb1dafc528 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:57.230344 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-820440e2-163a-4f1e-a97a-d1fb1dafc528 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '836c0e30f6ad4e289c90d2dcf0b60202'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:57.237574 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-820440e2-163a-4f1e-a97a-d1fb1dafc528 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '836c0e30f6ad4e289c90d2dcf0b60202'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:57.244474 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-820440e2-163a-4f1e-a97a-d1fb1dafc528 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '836c0e30f6ad4e289c90d2dcf0b60202'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:57.245137 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-820440e2-163a-4f1e-a97a-d1fb1dafc528 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '836c0e30f6ad4e289c90d2dcf0b60202'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:57.257685 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-820440e2-163a-4f1e-a97a-d1fb1dafc528 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '836c0e30f6ad4e289c90d2dcf0b60202'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:57.263597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1261/2519] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:19:57 2022] DELETE /identity/v3/users/836c0e30f6ad4e289c90d2dcf0b60202 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:57.268680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce998eb-dc61-4b8b-ab46-2fb0362db9d0 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:57.269167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce998eb-dc61-4b8b-ab46-2fb0362db9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:57.269287 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce998eb-dc61-4b8b-ab46-2fb0362db9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:57.286921 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-3ce998eb-dc61-4b8b-ab46-2fb0362db9d0 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:57.288203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3ce998eb-dc61-4b8b-ab46-2fb0362db9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:57.347962 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1259/2520] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:57.357608 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820440e2-163a-4f1e-a97a-d1fb1dafc528 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:57.359029 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-820440e2-163a-4f1e-a97a-d1fb1dafc528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:57.415984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820440e2-163a-4f1e-a97a-d1fb1dafc528 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:57.419892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d579b97-2f88-4ebf-b9a3-b3b00d656ac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:57.420563 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d579b97-2f88-4ebf-b9a3-b3b00d656ac7 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:57.420855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d579b97-2f88-4ebf-b9a3-b3b00d656ac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:57.421137 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d579b97-2f88-4ebf-b9a3-b3b00d656ac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:57.430111 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d579b97-2f88-4ebf-b9a3-b3b00d656ac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:57.431511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d579b97-2f88-4ebf-b9a3-b3b00d656ac7 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:57.441477 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1262/2521] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:57.713274 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce998eb-dc61-4b8b-ab46-2fb0362db9d0 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:57.714695 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3ce998eb-dc61-4b8b-ab46-2fb0362db9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:57.726797 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce998eb-dc61-4b8b-ab46-2fb0362db9d0 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:57.731290 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1faec73-e4ba-4449-9382-11f0771e0719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:57.732177 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faec73-e4ba-4449-9382-11f0771e0719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:57.732582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faec73-e4ba-4449-9382-11f0771e0719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:57.732867 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faec73-e4ba-4449-9382-11f0771e0719 admin admin] PATH_INFO: `/v3/projects/768200d8040e4cc4bf9c42c9aab5b73b` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:57.734075 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1faec73-e4ba-4449-9382-11f0771e0719 admin admin] RBAC: Authorizing `identity:delete_project(project_id=768200d8040e4cc4bf9c42c9aab5b73b)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:57.736592 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1faec73-e4ba-4449-9382-11f0771e0719 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:57.782831 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d1faec73-e4ba-4449-9382-11f0771e0719 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '768200d8040e4cc4bf9c42c9aab5b73b', 'request_id': 'req-d1faec73-e4ba-4449-9382-11f0771e0719'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:57.788611 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d1faec73-e4ba-4449-9382-11f0771e0719 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '768200d8040e4cc4bf9c42c9aab5b73b', 'request_id': 'req-d1faec73-e4ba-4449-9382-11f0771e0719'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:57.795852 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d1faec73-e4ba-4449-9382-11f0771e0719 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '768200d8040e4cc4bf9c42c9aab5b73b', 'request_id': 'req-d1faec73-e4ba-4449-9382-11f0771e0719'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:57.807633 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d1faec73-e4ba-4449-9382-11f0771e0719 admin admin] The token cache is being invalidate because project 768200d8040e4cc4bf9c42c9aab5b73b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:19:57.807922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d1faec73-e4ba-4449-9382-11f0771e0719 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:57.808577 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d1faec73-e4ba-4449-9382-11f0771e0719 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:57.810145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1260/2522] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:19:57 2022] DELETE /identity/v3/projects/768200d8040e4cc4bf9c42c9aab5b73b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:57.815779 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e1166-9872-40b0-b902-09f1a30e7e52 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:57.816121 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e1166-9872-40b0-b902-09f1a30e7e52 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:57.816406 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e1166-9872-40b0-b902-09f1a30e7e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:57.833375 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fd1e1166-9872-40b0-b902-09f1a30e7e52 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:57.834607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fd1e1166-9872-40b0-b902-09f1a30e7e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:57.851315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:57.894173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1263/2523] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:57.901631 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf346c6-281e-43e0-b47f-b8da6fdcb9f7 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:57.901921 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf346c6-281e-43e0-b47f-b8da6fdcb9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:57.902199 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf346c6-281e-43e0-b47f-b8da6fdcb9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:57.923246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-2cf346c6-281e-43e0-b47f-b8da6fdcb9f7 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:57.924506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2cf346c6-281e-43e0-b47f-b8da6fdcb9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:57.932213 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1261/2524] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:57.936244 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e1166-9872-40b0-b902-09f1a30e7e52 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:57.947243 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e1166-9872-40b0-b902-09f1a30e7e52 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:57.951851 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dbadb5-46cb-4f7a-a0c9-d97d8967aaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:57.952536 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dbadb5-46cb-4f7a-a0c9-d97d8967aaf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:57.952825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dbadb5-46cb-4f7a-a0c9-d97d8967aaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:57.953134 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dbadb5-46cb-4f7a-a0c9-d97d8967aaf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:57.953862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dbadb5-46cb-4f7a-a0c9-d97d8967aaf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:19:57.954176 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dbadb5-46cb-4f7a-a0c9-d97d8967aaf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:57.955778 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dbadb5-46cb-4f7a-a0c9-d97d8967aaf4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:57.967242 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1264/2525] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:19:57.971633 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf346c6-281e-43e0-b47f-b8da6fdcb9f7 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:57.982452 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf346c6-281e-43e0-b47f-b8da6fdcb9f7 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:57.987613 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ae0bf-3ad5-4d8b-bfa8-24997ebbe7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:57.988300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ae0bf-3ad5-4d8b-bfa8-24997ebbe7a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:57.988586 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ae0bf-3ad5-4d8b-bfa8-24997ebbe7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:57.988867 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ae0bf-3ad5-4d8b-bfa8-24997ebbe7a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:57.989567 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ae0bf-3ad5-4d8b-bfa8-24997ebbe7a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:57.991462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ae0bf-3ad5-4d8b-bfa8-24997ebbe7a6 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:58.017092 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1262/2526] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:19:57 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:58.021389 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dbadb5-46cb-4f7a-a0c9-d97d8967aaf4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:58.031719 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dbadb5-46cb-4f7a-a0c9-d97d8967aaf4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:58.087490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9c252-2222-4d97-b016-bcbcdb17733c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:58.088109 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9c252-2222-4d97-b016-bcbcdb17733c admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:58.088394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9c252-2222-4d97-b016-bcbcdb17733c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:58.088672 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9c252-2222-4d97-b016-bcbcdb17733c admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:58.089365 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc9c252-2222-4d97-b016-bcbcdb17733c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:58.091269 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc9c252-2222-4d97-b016-bcbcdb17733c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:58.110967 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1265/2527] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:19:58 2022] POST /identity/v3/users => generated 282 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:19:58.115038 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ae0bf-3ad5-4d8b-bfa8-24997ebbe7a6 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:58.125831 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ae0bf-3ad5-4d8b-bfa8-24997ebbe7a6 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:58.130410 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed7c62-1991-432b-a467-e22e3f1dd261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:58.131056 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed7c62-1991-432b-a467-e22e3f1dd261 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:58.131345 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed7c62-1991-432b-a467-e22e3f1dd261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:58.131625 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed7c62-1991-432b-a467-e22e3f1dd261 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:58.132387 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed7c62-1991-432b-a467-e22e3f1dd261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:58.133974 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed7c62-1991-432b-a467-e22e3f1dd261 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:58.144159 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1263/2528] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:58.147802 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9c252-2222-4d97-b016-bcbcdb17733c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:58.157577 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9c252-2222-4d97-b016-bcbcdb17733c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:58.162363 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf0788-a582-46b8-ba81-3b3ef4c85082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:58.163104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf0788-a582-46b8-ba81-3b3ef4c85082 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:58.163436 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf0788-a582-46b8-ba81-3b3ef4c85082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:58.163754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf0788-a582-46b8-ba81-3b3ef4c85082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:58.164521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abf0788-a582-46b8-ba81-3b3ef4c85082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:58.166271 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abf0788-a582-46b8-ba81-3b3ef4c85082 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:58.176852 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1266/2529] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:19:58.180755 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed7c62-1991-432b-a467-e22e3f1dd261 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:58.191193 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed7c62-1991-432b-a467-e22e3f1dd261 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:58.195796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c454af-40cb-4943-b923-f50b21a8a32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:58.196710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c454af-40cb-4943-b923-f50b21a8a32b admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:58.197021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c454af-40cb-4943-b923-f50b21a8a32b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:58.197302 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c454af-40cb-4943-b923-f50b21a8a32b admin admin] PATH_INFO: `/v3/projects/9c8e7aaf3c324aff9bcb7dc9f7efcf57/users/afab29f3614f4b439d52dda7f0514bd8/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:58.211560 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c454af-40cb-4943-b923-f50b21a8a32b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c8e7aaf3c324aff9bcb7dc9f7efcf57, user_id=afab29f3614f4b439d52dda7f0514bd8, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:58.214253 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c454af-40cb-4943-b923-f50b21a8a32b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:58.228449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1264/2530] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:58 2022] PUT /identity/v3/projects/9c8e7aaf3c324aff9bcb7dc9f7efcf57/users/afab29f3614f4b439d52dda7f0514bd8/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:19:58.240096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88dda0-fa8c-4506-90f0-ef1c529bf9f3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:58.240393 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88dda0-fa8c-4506-90f0-ef1c529bf9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:58.240674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88dda0-fa8c-4506-90f0-ef1c529bf9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:58.267401 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1d88dda0-fa8c-4506-90f0-ef1c529bf9f3 None None] MFA Rules not processed for user `afab29f3614f4b439d52dda7f0514bd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:58.268650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1d88dda0-fa8c-4506-90f0-ef1c529bf9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:58.334734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1267/2531] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:19:58 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:58.345793 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c454af-40cb-4943-b923-f50b21a8a32b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:58.347556 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-05c454af-40cb-4943-b923-f50b21a8a32b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:58.369895 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:58.407792 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c454af-40cb-4943-b923-f50b21a8a32b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:58.411452 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3436607-02a4-4604-8cab-8602e66ab238 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:58.412145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3436607-02a4-4604-8cab-8602e66ab238 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:58.412437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3436607-02a4-4604-8cab-8602e66ab238 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:58.412753 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3436607-02a4-4604-8cab-8602e66ab238 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:58.421527 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3436607-02a4-4604-8cab-8602e66ab238 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:58.423671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3436607-02a4-4604-8cab-8602e66ab238 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:58.432973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1265/2532] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:58 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:08.331553 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:08.428940 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=17 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:14.370262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d90f39-6c54-47ac-9370-ee95f591db01 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:14.370673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d90f39-6c54-47ac-9370-ee95f591db01 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:14.370833 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d90f39-6c54-47ac-9370-ee95f591db01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:14.388429 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-69d90f39-6c54-47ac-9370-ee95f591db01 None None] MFA Rules not processed for user `94d824bffc2c4af299c537056bc21734`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:14.389690 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-69d90f39-6c54-47ac-9370-ee95f591db01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:14.451561 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1268/2533] 10.222.0.94 () {58 vars in 967 bytes} [Mon Jul 18 14:20:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:14.474855 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3436607-02a4-4604-8cab-8602e66ab238 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:14.476770 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c3436607-02a4-4604-8cab-8602e66ab238 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:14.549468 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3436607-02a4-4604-8cab-8602e66ab238 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:14.554421 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222cb38-bbe3-40dd-a1a4-6bebcea590c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbf513ee25374e49b3d8ce28843e393a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:14.555070 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222cb38-bbe3-40dd-a1a4-6bebcea590c4 service swift] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:14.555385 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222cb38-bbe3-40dd-a1a4-6bebcea590c4 service swift] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:14.555664 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222cb38-bbe3-40dd-a1a4-6bebcea590c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:14.563614 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222cb38-bbe3-40dd-a1a4-6bebcea590c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:14.565238 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222cb38-bbe3-40dd-a1a4-6bebcea590c4 service swift] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:14.573209 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1266/2534] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Jul 18 14:20:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:20:16.251101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d90f39-6c54-47ac-9370-ee95f591db01 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:16.253332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-69d90f39-6c54-47ac-9370-ee95f591db01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:16.338737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d90f39-6c54-47ac-9370-ee95f591db01 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:16.345327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4413c72-b0cd-4ef9-9562-bf77a49d4eb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:16.346544 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4413c72-b0cd-4ef9-9562-bf77a49d4eb8 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:16.346986 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4413c72-b0cd-4ef9-9562-bf77a49d4eb8 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:16.347500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4413c72-b0cd-4ef9-9562-bf77a49d4eb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:16.349770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a4413c72-b0cd-4ef9-9562-bf77a49d4eb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:16.366591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4413c72-b0cd-4ef9-9562-bf77a49d4eb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:16.368523 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4413c72-b0cd-4ef9-9562-bf77a49d4eb8 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:16.385639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1269/2535] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:20:21.980165 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222cb38-bbe3-40dd-a1a4-6bebcea590c4 service swift] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:21.981565 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6222cb38-bbe3-40dd-a1a4-6bebcea590c4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:22.041299 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222cb38-bbe3-40dd-a1a4-6bebcea590c4 service swift] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:22.044894 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0840914-c419-4daf-82c0-993646a07b23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:22.045547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0840914-c419-4daf-82c0-993646a07b23 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:22.045836 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0840914-c419-4daf-82c0-993646a07b23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:22.046117 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0840914-c419-4daf-82c0-993646a07b23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:22.047912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d0840914-c419-4daf-82c0-993646a07b23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:22.113132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0840914-c419-4daf-82c0-993646a07b23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:22.114575 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0840914-c419-4daf-82c0-993646a07b23 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:22.125691 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1267/2536] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:25.326843 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4413c72-b0cd-4ef9-9562-bf77a49d4eb8 service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:25.327405 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a4413c72-b0cd-4ef9-9562-bf77a49d4eb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:25.397668 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0840914-c419-4daf-82c0-993646a07b23 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:25.398282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d0840914-c419-4daf-82c0-993646a07b23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:25.415374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0840914-c419-4daf-82c0-993646a07b23 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:25.419110 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4413c72-b0cd-4ef9-9562-bf77a49d4eb8 service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:25.420764 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4304a-cf76-40bc-86a3-5262577b280a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:25.421876 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4304a-cf76-40bc-86a3-5262577b280a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:25.422250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4304a-cf76-40bc-86a3-5262577b280a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:25.422584 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4304a-cf76-40bc-86a3-5262577b280a admin admin] PATH_INFO: `/v3/users/0542cd47ba77454ba25170292ed9aeb9` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:25.424206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d4304a-cf76-40bc-86a3-5262577b280a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0542cd47ba77454ba25170292ed9aeb9)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:25.425106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeea24f-934e-4246-83b7-f2f7a2b0119d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd5b2ba9e37041c78a99f1700a4970c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:25.425822 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeea24f-934e-4246-83b7-f2f7a2b0119d service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:25.426192 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeea24f-934e-4246-83b7-f2f7a2b0119d service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:25.426434 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d4304a-cf76-40bc-86a3-5262577b280a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:25.426571 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeea24f-934e-4246-83b7-f2f7a2b0119d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:25.429425 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-beeea24f-934e-4246-83b7-f2f7a2b0119d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:25.441451 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeea24f-934e-4246-83b7-f2f7a2b0119d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:25.443043 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeea24f-934e-4246-83b7-f2f7a2b0119d service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:25.458021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1270/2537] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:25.467098 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:25.480323 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-75d4304a-cf76-40bc-86a3-5262577b280a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0542cd47ba77454ba25170292ed9aeb9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:25.492340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-75d4304a-cf76-40bc-86a3-5262577b280a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0542cd47ba77454ba25170292ed9aeb9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:25.500446 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-75d4304a-cf76-40bc-86a3-5262577b280a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0542cd47ba77454ba25170292ed9aeb9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:25.501158 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-75d4304a-cf76-40bc-86a3-5262577b280a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0542cd47ba77454ba25170292ed9aeb9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:25.512901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-75d4304a-cf76-40bc-86a3-5262577b280a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0542cd47ba77454ba25170292ed9aeb9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:25.518793 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1268/2538] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:25 2022] DELETE /identity/v3/users/0542cd47ba77454ba25170292ed9aeb9 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:25.707081 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeea24f-934e-4246-83b7-f2f7a2b0119d service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:25.718369 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeea24f-934e-4246-83b7-f2f7a2b0119d service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:25.737100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:25.782724 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c797b5-431a-4415-a62f-07bf0114f56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:25.783608 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c797b5-431a-4415-a62f-07bf0114f56c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:25.783896 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c797b5-431a-4415-a62f-07bf0114f56c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:25.784212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c797b5-431a-4415-a62f-07bf0114f56c admin admin] PATH_INFO: `/v3/projects/26e4140751cf40c2adbfd92a29afd555` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:25.785441 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c797b5-431a-4415-a62f-07bf0114f56c admin admin] RBAC: Authorizing `identity:delete_project(project_id=26e4140751cf40c2adbfd92a29afd555)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:25.788845 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c797b5-431a-4415-a62f-07bf0114f56c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:25.841942 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d5c797b5-431a-4415-a62f-07bf0114f56c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26e4140751cf40c2adbfd92a29afd555', 'request_id': 'req-d5c797b5-431a-4415-a62f-07bf0114f56c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:25.848379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d5c797b5-431a-4415-a62f-07bf0114f56c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26e4140751cf40c2adbfd92a29afd555', 'request_id': 'req-d5c797b5-431a-4415-a62f-07bf0114f56c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:25.857532 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d5c797b5-431a-4415-a62f-07bf0114f56c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26e4140751cf40c2adbfd92a29afd555', 'request_id': 'req-d5c797b5-431a-4415-a62f-07bf0114f56c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:25.868118 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d5c797b5-431a-4415-a62f-07bf0114f56c admin admin] The token cache is being invalidate because project 26e4140751cf40c2adbfd92a29afd555 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:25.868501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d5c797b5-431a-4415-a62f-07bf0114f56c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:25.869234 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d5c797b5-431a-4415-a62f-07bf0114f56c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:25.870975 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1271/2539] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:25 2022] DELETE /identity/v3/projects/26e4140751cf40c2adbfd92a29afd555 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:25.876440 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286cd6d-ac0c-425a-9003-8e31d4dfc9f4 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:25.876959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286cd6d-ac0c-425a-9003-8e31d4dfc9f4 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:25.877318 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286cd6d-ac0c-425a-9003-8e31d4dfc9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:25.898124 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-3286cd6d-ac0c-425a-9003-8e31d4dfc9f4 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:25.899394 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3286cd6d-ac0c-425a-9003-8e31d4dfc9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:25.960312 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1269/2540] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:25.968095 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacc910-f118-4dd4-b14f-418af7a7f5fd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:25.968394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacc910-f118-4dd4-b14f-418af7a7f5fd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:25.968679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacc910-f118-4dd4-b14f-418af7a7f5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:25.988425 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6dacc910-f118-4dd4-b14f-418af7a7f5fd None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:25.989652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6dacc910-f118-4dd4-b14f-418af7a7f5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:25.997776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1272/2541] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:26.002011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286cd6d-ac0c-425a-9003-8e31d4dfc9f4 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:26.011706 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286cd6d-ac0c-425a-9003-8e31d4dfc9f4 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:26.016226 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccb3ec-2d02-4772-b475-852d6595cc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:26.016875 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccb3ec-2d02-4772-b475-852d6595cc85 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:26.017158 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccb3ec-2d02-4772-b475-852d6595cc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:26.017436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccb3ec-2d02-4772-b475-852d6595cc85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:26.018159 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ccb3ec-2d02-4772-b475-852d6595cc85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:20:26.018467 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ccb3ec-2d02-4772-b475-852d6595cc85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:26.020235 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ccb3ec-2d02-4772-b475-852d6595cc85 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:26.033781 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1270/2542] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:20:26.038808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dacc910-f118-4dd4-b14f-418af7a7f5fd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:26.047926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dacc910-f118-4dd4-b14f-418af7a7f5fd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:26.053024 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63a3e5-20eb-4762-85f1-7f01ae7bfea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:26.053691 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63a3e5-20eb-4762-85f1-7f01ae7bfea7 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:26.053981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63a3e5-20eb-4762-85f1-7f01ae7bfea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:26.054263 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63a3e5-20eb-4762-85f1-7f01ae7bfea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:26.054990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac63a3e5-20eb-4762-85f1-7f01ae7bfea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:26.056992 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac63a3e5-20eb-4762-85f1-7f01ae7bfea7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:26.083011 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1273/2543] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:26 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:20:26.087759 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccb3ec-2d02-4772-b475-852d6595cc85 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:26.097943 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccb3ec-2d02-4772-b475-852d6595cc85 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:26.157600 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73daf10c-8a98-475c-94c5-fd2857c5be69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:26.158185 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73daf10c-8a98-475c-94c5-fd2857c5be69 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:26.158477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73daf10c-8a98-475c-94c5-fd2857c5be69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:26.158760 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73daf10c-8a98-475c-94c5-fd2857c5be69 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:26.159492 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73daf10c-8a98-475c-94c5-fd2857c5be69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:26.161443 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73daf10c-8a98-475c-94c5-fd2857c5be69 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:26.181464 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1271/2544] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:26 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:26.185449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63a3e5-20eb-4762-85f1-7f01ae7bfea7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:26.198006 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63a3e5-20eb-4762-85f1-7f01ae7bfea7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:26.203212 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566637e1-27bf-4638-beb0-81876b9d300b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:26.204093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566637e1-27bf-4638-beb0-81876b9d300b admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:26.204387 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566637e1-27bf-4638-beb0-81876b9d300b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:26.204728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566637e1-27bf-4638-beb0-81876b9d300b admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:26.205463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566637e1-27bf-4638-beb0-81876b9d300b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:26.207128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566637e1-27bf-4638-beb0-81876b9d300b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:26.218648 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1274/2545] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:26.222542 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73daf10c-8a98-475c-94c5-fd2857c5be69 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:26.232903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73daf10c-8a98-475c-94c5-fd2857c5be69 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:26.237697 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f5d34-a0dc-4992-8daf-5a6add454085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:26.238460 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f5d34-a0dc-4992-8daf-5a6add454085 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:26.238751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f5d34-a0dc-4992-8daf-5a6add454085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:26.239021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f5d34-a0dc-4992-8daf-5a6add454085 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:26.239862 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f5d34-a0dc-4992-8daf-5a6add454085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:26.241998 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f5d34-a0dc-4992-8daf-5a6add454085 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:26.253345 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1272/2546] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:26.258262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566637e1-27bf-4638-beb0-81876b9d300b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:26.268339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566637e1-27bf-4638-beb0-81876b9d300b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:26.273718 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca75ae3-065e-4517-8b0b-47cc1b3721c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:26.274593 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca75ae3-065e-4517-8b0b-47cc1b3721c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:26.274885 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca75ae3-065e-4517-8b0b-47cc1b3721c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:26.275172 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca75ae3-065e-4517-8b0b-47cc1b3721c5 admin admin] PATH_INFO: `/v3/projects/e48020d9a6d34cb6876ae43c4cf0264b/users/597aa04d29b84f1aae4553ea4d4d5c32/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:26.289379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca75ae3-065e-4517-8b0b-47cc1b3721c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e48020d9a6d34cb6876ae43c4cf0264b, user_id=597aa04d29b84f1aae4553ea4d4d5c32, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:26.292170 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca75ae3-065e-4517-8b0b-47cc1b3721c5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:26.306548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1275/2547] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:26 2022] PUT /identity/v3/projects/e48020d9a6d34cb6876ae43c4cf0264b/users/597aa04d29b84f1aae4553ea4d4d5c32/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:26.317478 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64305d2c-d5bf-4a0c-a08e-f39990885032 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:26.317769 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64305d2c-d5bf-4a0c-a08e-f39990885032 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:26.318048 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64305d2c-d5bf-4a0c-a08e-f39990885032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:26.346169 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-64305d2c-d5bf-4a0c-a08e-f39990885032 None None] MFA Rules not processed for user `597aa04d29b84f1aae4553ea4d4d5c32`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:26.347453 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-64305d2c-d5bf-4a0c-a08e-f39990885032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:26.353848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:26.417630 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1273/2548] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:26 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:26.421937 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca75ae3-065e-4517-8b0b-47cc1b3721c5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:26.470761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:26.486904 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca75ae3-065e-4517-8b0b-47cc1b3721c5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:26.492194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39cd530-9171-4fdd-b92b-50962f2a68a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:26.492862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39cd530-9171-4fdd-b92b-50962f2a68a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:26.493154 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39cd530-9171-4fdd-b92b-50962f2a68a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:26.493435 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39cd530-9171-4fdd-b92b-50962f2a68a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:26.494149 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39cd530-9171-4fdd-b92b-50962f2a68a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:26.496094 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39cd530-9171-4fdd-b92b-50962f2a68a9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:26.522232 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1276/2549] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:26 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:26.526333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64305d2c-d5bf-4a0c-a08e-f39990885032 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:26.537151 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64305d2c-d5bf-4a0c-a08e-f39990885032 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:26.594737 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be2691-add4-40e0-906e-626c3bfa3142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:26.595422 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be2691-add4-40e0-906e-626c3bfa3142 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:26.595713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be2691-add4-40e0-906e-626c3bfa3142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:26.596037 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be2691-add4-40e0-906e-626c3bfa3142 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:26.596751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be2691-add4-40e0-906e-626c3bfa3142 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:26.598687 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be2691-add4-40e0-906e-626c3bfa3142 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:26.619296 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1274/2550] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:26 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:26.623260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39cd530-9171-4fdd-b92b-50962f2a68a9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:26.634517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39cd530-9171-4fdd-b92b-50962f2a68a9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:26.639532 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630dcb0d-fbf6-455c-839b-b722ec158e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:26.640245 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630dcb0d-fbf6-455c-839b-b722ec158e02 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:26.640539 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630dcb0d-fbf6-455c-839b-b722ec158e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:26.640822 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630dcb0d-fbf6-455c-839b-b722ec158e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:26.641564 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630dcb0d-fbf6-455c-839b-b722ec158e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:26.643173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630dcb0d-fbf6-455c-839b-b722ec158e02 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:26.654725 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1277/2551] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:26.658892 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be2691-add4-40e0-906e-626c3bfa3142 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:26.669326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be2691-add4-40e0-906e-626c3bfa3142 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:26.675089 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa8ee1-fd80-4811-a366-cd5415481888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:26.676107 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa8ee1-fd80-4811-a366-cd5415481888 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:26.676412 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa8ee1-fd80-4811-a366-cd5415481888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:26.676697 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa8ee1-fd80-4811-a366-cd5415481888 admin admin] PATH_INFO: `/v3/projects/eb1c5ec7878f41dfbde083bed80414df/users/f7ecbf47f7da4190b9b300489c8abe38/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:26.692643 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa8ee1-fd80-4811-a366-cd5415481888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb1c5ec7878f41dfbde083bed80414df, user_id=f7ecbf47f7da4190b9b300489c8abe38, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:26.696003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa8ee1-fd80-4811-a366-cd5415481888 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:26.710089 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1275/2552] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:26 2022] PUT /identity/v3/projects/eb1c5ec7878f41dfbde083bed80414df/users/f7ecbf47f7da4190b9b300489c8abe38/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:26.714316 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630dcb0d-fbf6-455c-839b-b722ec158e02 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:26.724809 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630dcb0d-fbf6-455c-839b-b722ec158e02 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:26.783695 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37818a-3291-4f25-9685-1f66bc40e511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:26.784393 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37818a-3291-4f25-9685-1f66bc40e511 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:26.784680 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37818a-3291-4f25-9685-1f66bc40e511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:26.784958 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37818a-3291-4f25-9685-1f66bc40e511 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:26.785679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37818a-3291-4f25-9685-1f66bc40e511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:26.787270 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37818a-3291-4f25-9685-1f66bc40e511 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:26.797508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1278/2553] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:26.801974 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa8ee1-fd80-4811-a366-cd5415481888 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:26.813072 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa8ee1-fd80-4811-a366-cd5415481888 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:26.817669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01dddb0-1319-4818-b195-140f05f9a807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:26.818585 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01dddb0-1319-4818-b195-140f05f9a807 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:26.818871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01dddb0-1319-4818-b195-140f05f9a807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:26.819836 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01dddb0-1319-4818-b195-140f05f9a807 admin admin] PATH_INFO: `/v3/projects/eb1c5ec7878f41dfbde083bed80414df/users/f7ecbf47f7da4190b9b300489c8abe38/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:26.821844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01dddb0-1319-4818-b195-140f05f9a807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb1c5ec7878f41dfbde083bed80414df, user_id=f7ecbf47f7da4190b9b300489c8abe38, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:26.824554 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01dddb0-1319-4818-b195-140f05f9a807 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:26.835110 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1276/2554] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:26 2022] PUT /identity/v3/projects/eb1c5ec7878f41dfbde083bed80414df/users/f7ecbf47f7da4190b9b300489c8abe38/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:26.844977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52584d-24a1-47a0-bdab-2a5a7a2cdac5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:26.845335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52584d-24a1-47a0-bdab-2a5a7a2cdac5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:26.845626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52584d-24a1-47a0-bdab-2a5a7a2cdac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:26.872754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4f52584d-24a1-47a0-bdab-2a5a7a2cdac5 None None] MFA Rules not processed for user `f7ecbf47f7da4190b9b300489c8abe38`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:26.874058 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4f52584d-24a1-47a0-bdab-2a5a7a2cdac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:26.943757 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1279/2555] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:26 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:26.954983 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01dddb0-1319-4818-b195-140f05f9a807 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:26.956736 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f01dddb0-1319-4818-b195-140f05f9a807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:27.020402 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01dddb0-1319-4818-b195-140f05f9a807 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:27.024156 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcaa525-573f-4387-8a52-13bea88b74bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:27.024847 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcaa525-573f-4387-8a52-13bea88b74bd service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:27.025139 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcaa525-573f-4387-8a52-13bea88b74bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:27.025424 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcaa525-573f-4387-8a52-13bea88b74bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:27.034512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcaa525-573f-4387-8a52-13bea88b74bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:27.035925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcaa525-573f-4387-8a52-13bea88b74bd service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:27.047892 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1277/2556] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:26 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:27.119418 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52584d-24a1-47a0-bdab-2a5a7a2cdac5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:27.129317 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52584d-24a1-47a0-bdab-2a5a7a2cdac5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:27.133258 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc2aae-5458-4905-9c9c-5826999ae4a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:27.133955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc2aae-5458-4905-9c9c-5826999ae4a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:27.134310 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc2aae-5458-4905-9c9c-5826999ae4a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:27.134618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc2aae-5458-4905-9c9c-5826999ae4a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:27.176078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:27.194431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc2aae-5458-4905-9c9c-5826999ae4a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:27.195814 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc2aae-5458-4905-9c9c-5826999ae4a8 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:27.205109 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1280/2557] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:27 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:29.903973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcaa525-573f-4387-8a52-13bea88b74bd service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:29.908345 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:29.969028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcaa525-573f-4387-8a52-13bea88b74bd service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:29.973640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520c91d-553e-4623-9db6-03ebde9e66cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:29.974516 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520c91d-553e-4623-9db6-03ebde9e66cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:29.974803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520c91d-553e-4623-9db6-03ebde9e66cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:29.975083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520c91d-553e-4623-9db6-03ebde9e66cb admin admin] PATH_INFO: `/v3/users/597aa04d29b84f1aae4553ea4d4d5c32` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:29.976306 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5520c91d-553e-4623-9db6-03ebde9e66cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=597aa04d29b84f1aae4553ea4d4d5c32)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:29.978240 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5520c91d-553e-4623-9db6-03ebde9e66cb admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:30.029132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5520c91d-553e-4623-9db6-03ebde9e66cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '597aa04d29b84f1aae4553ea4d4d5c32'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:30.035985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5520c91d-553e-4623-9db6-03ebde9e66cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '597aa04d29b84f1aae4553ea4d4d5c32'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:30.042944 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5520c91d-553e-4623-9db6-03ebde9e66cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '597aa04d29b84f1aae4553ea4d4d5c32'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:30.043577 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5520c91d-553e-4623-9db6-03ebde9e66cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '597aa04d29b84f1aae4553ea4d4d5c32'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:30.055424 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5520c91d-553e-4623-9db6-03ebde9e66cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '597aa04d29b84f1aae4553ea4d4d5c32'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:30.061288 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1278/2558] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:29 2022] DELETE /identity/v3/users/597aa04d29b84f1aae4553ea4d4d5c32 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:30.065923 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909c0cb-7cbf-4eda-abe7-55c813937b2e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:30.066417 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909c0cb-7cbf-4eda-abe7-55c813937b2e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:30.066527 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909c0cb-7cbf-4eda-abe7-55c813937b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:30.083597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5909c0cb-7cbf-4eda-abe7-55c813937b2e None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:30.084849 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5909c0cb-7cbf-4eda-abe7-55c813937b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:30.144675 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1281/2559] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:30.155170 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520c91d-553e-4623-9db6-03ebde9e66cb admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:30.156618 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5520c91d-553e-4623-9db6-03ebde9e66cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:30.213639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520c91d-553e-4623-9db6-03ebde9e66cb admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:30.217241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481c283-0a14-4e2c-b212-68c53a25b53a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:30.217886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481c283-0a14-4e2c-b212-68c53a25b53a service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:30.218172 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481c283-0a14-4e2c-b212-68c53a25b53a service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:30.218450 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481c283-0a14-4e2c-b212-68c53a25b53a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:30.227269 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481c283-0a14-4e2c-b212-68c53a25b53a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:30.228726 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481c283-0a14-4e2c-b212-68c53a25b53a service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:30.239084 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1279/2560] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:30.499389 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5909c0cb-7cbf-4eda-abe7-55c813937b2e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:30.500913 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5909c0cb-7cbf-4eda-abe7-55c813937b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:30.514527 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5909c0cb-7cbf-4eda-abe7-55c813937b2e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:30.519862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652bce0-af9e-4013-8f08-da41a8c3f9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:30.520765 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652bce0-af9e-4013-8f08-da41a8c3f9b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:30.521050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652bce0-af9e-4013-8f08-da41a8c3f9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:30.521328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652bce0-af9e-4013-8f08-da41a8c3f9b3 admin admin] PATH_INFO: `/v3/projects/e48020d9a6d34cb6876ae43c4cf0264b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:30.522538 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b652bce0-af9e-4013-8f08-da41a8c3f9b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e48020d9a6d34cb6876ae43c4cf0264b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:30.524507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b652bce0-af9e-4013-8f08-da41a8c3f9b3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:30.574754 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b652bce0-af9e-4013-8f08-da41a8c3f9b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e48020d9a6d34cb6876ae43c4cf0264b', 'request_id': 'req-b652bce0-af9e-4013-8f08-da41a8c3f9b3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:30.580918 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b652bce0-af9e-4013-8f08-da41a8c3f9b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e48020d9a6d34cb6876ae43c4cf0264b', 'request_id': 'req-b652bce0-af9e-4013-8f08-da41a8c3f9b3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:30.589396 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b652bce0-af9e-4013-8f08-da41a8c3f9b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e48020d9a6d34cb6876ae43c4cf0264b', 'request_id': 'req-b652bce0-af9e-4013-8f08-da41a8c3f9b3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:30.599483 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b652bce0-af9e-4013-8f08-da41a8c3f9b3 admin admin] The token cache is being invalidate because project e48020d9a6d34cb6876ae43c4cf0264b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:30.599773 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b652bce0-af9e-4013-8f08-da41a8c3f9b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:30.600447 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b652bce0-af9e-4013-8f08-da41a8c3f9b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:30.602498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1282/2561] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:30 2022] DELETE /identity/v3/projects/e48020d9a6d34cb6876ae43c4cf0264b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:30.606425 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481c283-0a14-4e2c-b212-68c53a25b53a service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:30.616004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481c283-0a14-4e2c-b212-68c53a25b53a service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:30.635821 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:30.674013 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afc7c7-7ad0-434a-9a65-91e3d4e05e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:30.675140 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afc7c7-7ad0-434a-9a65-91e3d4e05e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:30.675426 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afc7c7-7ad0-434a-9a65-91e3d4e05e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:30.675707 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afc7c7-7ad0-434a-9a65-91e3d4e05e6b admin admin] PATH_INFO: `/v3/users/f7ecbf47f7da4190b9b300489c8abe38` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:30.676955 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afc7c7-7ad0-434a-9a65-91e3d4e05e6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7ecbf47f7da4190b9b300489c8abe38)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:30.678916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afc7c7-7ad0-434a-9a65-91e3d4e05e6b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:30.725284 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-57afc7c7-7ad0-434a-9a65-91e3d4e05e6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7ecbf47f7da4190b9b300489c8abe38'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:30.733095 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-57afc7c7-7ad0-434a-9a65-91e3d4e05e6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7ecbf47f7da4190b9b300489c8abe38'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:30.740360 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-57afc7c7-7ad0-434a-9a65-91e3d4e05e6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7ecbf47f7da4190b9b300489c8abe38'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:30.740992 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-57afc7c7-7ad0-434a-9a65-91e3d4e05e6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7ecbf47f7da4190b9b300489c8abe38'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:30.753920 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-57afc7c7-7ad0-434a-9a65-91e3d4e05e6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7ecbf47f7da4190b9b300489c8abe38'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:30.760131 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1280/2562] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:30 2022] DELETE /identity/v3/users/f7ecbf47f7da4190b9b300489c8abe38 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:31.040085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652bce0-af9e-4013-8f08-da41a8c3f9b3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:31.041530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b652bce0-af9e-4013-8f08-da41a8c3f9b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:31.056646 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652bce0-af9e-4013-8f08-da41a8c3f9b3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:31.112345 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954a3f8-edba-4203-9a5b-fb0f73c08d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:31.113135 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954a3f8-edba-4203-9a5b-fb0f73c08d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:31.113421 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954a3f8-edba-4203-9a5b-fb0f73c08d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:31.113697 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954a3f8-edba-4203-9a5b-fb0f73c08d84 admin admin] PATH_INFO: `/v3/projects/eb1c5ec7878f41dfbde083bed80414df` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:31.114863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954a3f8-edba-4203-9a5b-fb0f73c08d84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb1c5ec7878f41dfbde083bed80414df)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:31.116828 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954a3f8-edba-4203-9a5b-fb0f73c08d84 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:31.122806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:31.164480 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b954a3f8-edba-4203-9a5b-fb0f73c08d84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb1c5ec7878f41dfbde083bed80414df', 'request_id': 'req-b954a3f8-edba-4203-9a5b-fb0f73c08d84'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:31.170465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b954a3f8-edba-4203-9a5b-fb0f73c08d84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb1c5ec7878f41dfbde083bed80414df', 'request_id': 'req-b954a3f8-edba-4203-9a5b-fb0f73c08d84'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:31.177546 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b954a3f8-edba-4203-9a5b-fb0f73c08d84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb1c5ec7878f41dfbde083bed80414df', 'request_id': 'req-b954a3f8-edba-4203-9a5b-fb0f73c08d84'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:31.187578 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b954a3f8-edba-4203-9a5b-fb0f73c08d84 admin admin] The token cache is being invalidate because project eb1c5ec7878f41dfbde083bed80414df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:31.187863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b954a3f8-edba-4203-9a5b-fb0f73c08d84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:31.188521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b954a3f8-edba-4203-9a5b-fb0f73c08d84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:31.190035 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1283/2563] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:31 2022] DELETE /identity/v3/projects/eb1c5ec7878f41dfbde083bed80414df => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:31.195045 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea8c51-1d6d-4770-8dc4-04efcb7e0cc6 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:31.195508 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea8c51-1d6d-4770-8dc4-04efcb7e0cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:31.195605 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea8c51-1d6d-4770-8dc4-04efcb7e0cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:31.213345 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-8aea8c51-1d6d-4770-8dc4-04efcb7e0cc6 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:31.214553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8aea8c51-1d6d-4770-8dc4-04efcb7e0cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:31.275655 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1281/2564] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:20:31.283335 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570f732-9eee-4662-b181-01cbcb330535 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:31.283629 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570f732-9eee-4662-b181-01cbcb330535 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:31.283914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570f732-9eee-4662-b181-01cbcb330535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:31.302584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f570f732-9eee-4662-b181-01cbcb330535 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:31.303784 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f570f732-9eee-4662-b181-01cbcb330535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:31.311647 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1284/2565] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:31.315610 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea8c51-1d6d-4770-8dc4-04efcb7e0cc6 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:31.327998 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea8c51-1d6d-4770-8dc4-04efcb7e0cc6 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:31.333470 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53c047-c8fd-4452-931e-051b002abb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:31.334128 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53c047-c8fd-4452-931e-051b002abb8e admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:31.334431 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53c047-c8fd-4452-931e-051b002abb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:31.334704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e53c047-c8fd-4452-931e-051b002abb8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:31.335432 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53c047-c8fd-4452-931e-051b002abb8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:20:31.336327 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53c047-c8fd-4452-931e-051b002abb8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:31.337972 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e53c047-c8fd-4452-931e-051b002abb8e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:31.349541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1282/2566] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:20:31.353881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570f732-9eee-4662-b181-01cbcb330535 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:31.363081 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570f732-9eee-4662-b181-01cbcb330535 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:31.367699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eaf8e2-b8c5-46c6-8c93-2bde9fe9de9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:31.368386 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eaf8e2-b8c5-46c6-8c93-2bde9fe9de9f admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:31.368672 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eaf8e2-b8c5-46c6-8c93-2bde9fe9de9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:31.368951 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eaf8e2-b8c5-46c6-8c93-2bde9fe9de9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:31.369639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eaf8e2-b8c5-46c6-8c93-2bde9fe9de9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:31.371540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eaf8e2-b8c5-46c6-8c93-2bde9fe9de9f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:31.396208 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1285/2567] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:31 2022] POST /identity/v3/projects => generated 379 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:31.401049 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53c047-c8fd-4452-931e-051b002abb8e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:31.410986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e53c047-c8fd-4452-931e-051b002abb8e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:31.439640 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:31.470133 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191943f3-4bdd-4c75-8f79-15cd4808ffec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:31.470803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191943f3-4bdd-4c75-8f79-15cd4808ffec admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:31.471118 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191943f3-4bdd-4c75-8f79-15cd4808ffec admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:31.471387 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191943f3-4bdd-4c75-8f79-15cd4808ffec admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:31.472126 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191943f3-4bdd-4c75-8f79-15cd4808ffec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:31.474046 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191943f3-4bdd-4c75-8f79-15cd4808ffec admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:31.493057 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1283/2568] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:31 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:31.497191 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eaf8e2-b8c5-46c6-8c93-2bde9fe9de9f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:31.507856 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eaf8e2-b8c5-46c6-8c93-2bde9fe9de9f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:31.512550 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5f996-d182-49e3-911b-48256ae4c3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:31.513245 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5f996-d182-49e3-911b-48256ae4c3c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:31.513538 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5f996-d182-49e3-911b-48256ae4c3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:31.513826 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5f996-d182-49e3-911b-48256ae4c3c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:31.514587 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c5f996-d182-49e3-911b-48256ae4c3c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:31.516261 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c5f996-d182-49e3-911b-48256ae4c3c2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:31.527275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1286/2569] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:31.531004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191943f3-4bdd-4c75-8f79-15cd4808ffec admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:31.540947 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191943f3-4bdd-4c75-8f79-15cd4808ffec admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:31.545612 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f4906-5aef-4ad3-8930-c1dd4edc9e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:31.546266 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f4906-5aef-4ad3-8930-c1dd4edc9e98 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:31.546558 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f4906-5aef-4ad3-8930-c1dd4edc9e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:31.546859 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f4906-5aef-4ad3-8930-c1dd4edc9e98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:31.547572 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f4906-5aef-4ad3-8930-c1dd4edc9e98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:31.549207 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f4906-5aef-4ad3-8930-c1dd4edc9e98 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:31.559248 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1284/2570] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:31.563055 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5f996-d182-49e3-911b-48256ae4c3c2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:31.573501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5f996-d182-49e3-911b-48256ae4c3c2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:31.578501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6eb51-3656-44fa-8ead-b9de48b0e9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:31.579413 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6eb51-3656-44fa-8ead-b9de48b0e9b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:31.579705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6eb51-3656-44fa-8ead-b9de48b0e9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:31.580012 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6eb51-3656-44fa-8ead-b9de48b0e9b8 admin admin] PATH_INFO: `/v3/projects/d937813020b04a3eb724e373664872c3/users/8917b39ce5ab4e3ea2bfb846a1e361e9/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:31.596937 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e6eb51-3656-44fa-8ead-b9de48b0e9b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d937813020b04a3eb724e373664872c3, user_id=8917b39ce5ab4e3ea2bfb846a1e361e9, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:31.599429 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e6eb51-3656-44fa-8ead-b9de48b0e9b8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:31.619865 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1287/2571] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:31 2022] PUT /identity/v3/projects/d937813020b04a3eb724e373664872c3/users/8917b39ce5ab4e3ea2bfb846a1e361e9/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:20:31.627395 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6629d03-dce7-4ed4-96aa-b7a5c4e0c9ae None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:31.627759 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6629d03-dce7-4ed4-96aa-b7a5c4e0c9ae None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:31.629238 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6629d03-dce7-4ed4-96aa-b7a5c4e0c9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:31.655242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d6629d03-dce7-4ed4-96aa-b7a5c4e0c9ae None None] MFA Rules not processed for user `8917b39ce5ab4e3ea2bfb846a1e361e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:31.656490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d6629d03-dce7-4ed4-96aa-b7a5c4e0c9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:31.719220 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1285/2572] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:31.723653 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6eb51-3656-44fa-8ead-b9de48b0e9b8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:31.784748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6eb51-3656-44fa-8ead-b9de48b0e9b8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:31.789923 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d506a8-2b74-44a0-af68-9a197efa682b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:31.790574 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d506a8-2b74-44a0-af68-9a197efa682b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:31.790861 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d506a8-2b74-44a0-af68-9a197efa682b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:31.791142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d506a8-2b74-44a0-af68-9a197efa682b admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:31.791837 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d506a8-2b74-44a0-af68-9a197efa682b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:31.793781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d506a8-2b74-44a0-af68-9a197efa682b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:31.817728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1288/2573] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:31 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:31.821761 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6629d03-dce7-4ed4-96aa-b7a5c4e0c9ae None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:31.832494 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6629d03-dce7-4ed4-96aa-b7a5c4e0c9ae None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:31.887898 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679051fa-8738-4c1e-8a7f-8aaf4ce44e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:31.888591 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679051fa-8738-4c1e-8a7f-8aaf4ce44e8c admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:31.888880 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679051fa-8738-4c1e-8a7f-8aaf4ce44e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:31.889165 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679051fa-8738-4c1e-8a7f-8aaf4ce44e8c admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:31.889847 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679051fa-8738-4c1e-8a7f-8aaf4ce44e8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:31.891771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679051fa-8738-4c1e-8a7f-8aaf4ce44e8c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:31.910998 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1286/2574] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:31 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:31.915480 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d506a8-2b74-44a0-af68-9a197efa682b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:31.925976 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d506a8-2b74-44a0-af68-9a197efa682b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:31.930863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b37bf1-4209-475d-ab30-a681640ebb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:31.931532 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b37bf1-4209-475d-ab30-a681640ebb43 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:31.931819 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b37bf1-4209-475d-ab30-a681640ebb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:31.932136 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b37bf1-4209-475d-ab30-a681640ebb43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:31.932862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b37bf1-4209-475d-ab30-a681640ebb43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:31.934495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b37bf1-4209-475d-ab30-a681640ebb43 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:31.945767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1289/2575] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:20:31.949654 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679051fa-8738-4c1e-8a7f-8aaf4ce44e8c admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:31.964590 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679051fa-8738-4c1e-8a7f-8aaf4ce44e8c admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:31.969935 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2ef4f-5986-4dcd-8820-bf7caee234aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:31.970659 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2ef4f-5986-4dcd-8820-bf7caee234aa admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:31.970948 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2ef4f-5986-4dcd-8820-bf7caee234aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:31.971231 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2ef4f-5986-4dcd-8820-bf7caee234aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:31.971988 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2ef4f-5986-4dcd-8820-bf7caee234aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:31.973625 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2ef4f-5986-4dcd-8820-bf7caee234aa admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:31.984606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1287/2576] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:20:31.988826 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b37bf1-4209-475d-ab30-a681640ebb43 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:31.995151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:31.998881 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b37bf1-4209-475d-ab30-a681640ebb43 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:32.004506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51578f6-68d5-46c4-8e23-fef02d76e57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:32.005545 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51578f6-68d5-46c4-8e23-fef02d76e57c admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:32.005998 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51578f6-68d5-46c4-8e23-fef02d76e57c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:32.006459 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51578f6-68d5-46c4-8e23-fef02d76e57c admin admin] PATH_INFO: `/v3/projects/a857efebf62b40d187170bbe6f2ca149/users/ed4cf2446adb4b1db35266433cfdcb22/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:32.020700 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51578f6-68d5-46c4-8e23-fef02d76e57c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a857efebf62b40d187170bbe6f2ca149, user_id=ed4cf2446adb4b1db35266433cfdcb22, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:32.023903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51578f6-68d5-46c4-8e23-fef02d76e57c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:32.036585 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1290/2577] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:31 2022] PUT /identity/v3/projects/a857efebf62b40d187170bbe6f2ca149/users/ed4cf2446adb4b1db35266433cfdcb22/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:32.048036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a556276-bf59-4642-9ad6-16011565a824 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:32.048336 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a556276-bf59-4642-9ad6-16011565a824 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:32.048639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a556276-bf59-4642-9ad6-16011565a824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:32.074797 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-6a556276-bf59-4642-9ad6-16011565a824 None None] MFA Rules not processed for user `ed4cf2446adb4b1db35266433cfdcb22`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:32.076039 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6a556276-bf59-4642-9ad6-16011565a824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:32.080922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:32.138858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1288/2578] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:20:32.143349 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51578f6-68d5-46c4-8e23-fef02d76e57c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:32.204627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51578f6-68d5-46c4-8e23-fef02d76e57c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:32.209287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947456ef-aced-46f8-bcbe-0aeca87fca3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:32.209973 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947456ef-aced-46f8-bcbe-0aeca87fca3a admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:32.210265 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947456ef-aced-46f8-bcbe-0aeca87fca3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:32.210602 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947456ef-aced-46f8-bcbe-0aeca87fca3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:32.211861 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947456ef-aced-46f8-bcbe-0aeca87fca3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:32.213803 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947456ef-aced-46f8-bcbe-0aeca87fca3a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:32.238575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1291/2579] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:32 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:32.242652 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a556276-bf59-4642-9ad6-16011565a824 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:32.253536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a556276-bf59-4642-9ad6-16011565a824 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:32.309463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61649a25-06b6-46b9-bfd7-49a630e367d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:32.310142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61649a25-06b6-46b9-bfd7-49a630e367d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:32.310429 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61649a25-06b6-46b9-bfd7-49a630e367d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:32.310710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61649a25-06b6-46b9-bfd7-49a630e367d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:32.311406 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61649a25-06b6-46b9-bfd7-49a630e367d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:32.313403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61649a25-06b6-46b9-bfd7-49a630e367d0 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:32.333599 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1289/2580] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:32 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:32.338232 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947456ef-aced-46f8-bcbe-0aeca87fca3a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:32.348573 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947456ef-aced-46f8-bcbe-0aeca87fca3a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:32.353360 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efcae8e-f491-45e6-9d7c-7275318c583f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:32.354013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efcae8e-f491-45e6-9d7c-7275318c583f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:32.354299 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efcae8e-f491-45e6-9d7c-7275318c583f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:32.354579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efcae8e-f491-45e6-9d7c-7275318c583f admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:32.355293 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efcae8e-f491-45e6-9d7c-7275318c583f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:32.356919 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efcae8e-f491-45e6-9d7c-7275318c583f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:32.368295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1292/2581] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:32.372176 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61649a25-06b6-46b9-bfd7-49a630e367d0 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:32.382029 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61649a25-06b6-46b9-bfd7-49a630e367d0 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:32.387087 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698f7d6-6880-4d89-899f-54da05e79420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:32.387982 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698f7d6-6880-4d89-899f-54da05e79420 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:32.388266 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698f7d6-6880-4d89-899f-54da05e79420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:32.388548 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698f7d6-6880-4d89-899f-54da05e79420 admin admin] PATH_INFO: `/v3/projects/af86cae93be44cb1aa17b096cf369723/users/2f7a14fc635c40ea8b2297c33ea19335/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:32.403445 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698f7d6-6880-4d89-899f-54da05e79420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af86cae93be44cb1aa17b096cf369723, user_id=2f7a14fc635c40ea8b2297c33ea19335, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:32.406067 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698f7d6-6880-4d89-899f-54da05e79420 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:32.418483 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1290/2582] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:32 2022] PUT /identity/v3/projects/af86cae93be44cb1aa17b096cf369723/users/2f7a14fc635c40ea8b2297c33ea19335/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:32.422439 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efcae8e-f491-45e6-9d7c-7275318c583f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:32.432678 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efcae8e-f491-45e6-9d7c-7275318c583f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:32.487450 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838343f9-4998-4dfd-8668-4cf0901cc12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:32.488146 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838343f9-4998-4dfd-8668-4cf0901cc12e admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:32.488458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838343f9-4998-4dfd-8668-4cf0901cc12e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:32.488744 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838343f9-4998-4dfd-8668-4cf0901cc12e admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:32.489464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838343f9-4998-4dfd-8668-4cf0901cc12e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:32.491098 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838343f9-4998-4dfd-8668-4cf0901cc12e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:32.501745 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1293/2583] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:32.505878 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698f7d6-6880-4d89-899f-54da05e79420 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:32.516751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698f7d6-6880-4d89-899f-54da05e79420 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:32.521273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d1e6b-2edb-4bfa-ae16-08a5d8ce907d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:32.522144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d1e6b-2edb-4bfa-ae16-08a5d8ce907d admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:32.522429 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d1e6b-2edb-4bfa-ae16-08a5d8ce907d admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:32.522709 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d1e6b-2edb-4bfa-ae16-08a5d8ce907d admin admin] PATH_INFO: `/v3/projects/af86cae93be44cb1aa17b096cf369723/users/2f7a14fc635c40ea8b2297c33ea19335/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:32.524694 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d1e6b-2edb-4bfa-ae16-08a5d8ce907d admin admin] RBAC: Authorizing `identity:create_grant(project_id=af86cae93be44cb1aa17b096cf369723, user_id=2f7a14fc635c40ea8b2297c33ea19335, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:32.527332 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d1e6b-2edb-4bfa-ae16-08a5d8ce907d admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:32.537390 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1291/2584] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:32 2022] PUT /identity/v3/projects/af86cae93be44cb1aa17b096cf369723/users/2f7a14fc635c40ea8b2297c33ea19335/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:32.548162 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362d96c-f88b-47fb-b0a6-718fed1d7d2b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:32.548462 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362d96c-f88b-47fb-b0a6-718fed1d7d2b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:32.548746 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362d96c-f88b-47fb-b0a6-718fed1d7d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:32.575515 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4362d96c-f88b-47fb-b0a6-718fed1d7d2b None None] MFA Rules not processed for user `2f7a14fc635c40ea8b2297c33ea19335`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:32.576764 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4362d96c-f88b-47fb-b0a6-718fed1d7d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:32.615211 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:32.647202 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1294/2585] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:32.658576 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d1e6b-2edb-4bfa-ae16-08a5d8ce907d admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:32.660305 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ab4d1e6b-2edb-4bfa-ae16-08a5d8ce907d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:32.716991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d1e6b-2edb-4bfa-ae16-08a5d8ce907d admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:32.720691 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0add84-1da1-497d-b34d-583796c8f1f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:32.721335 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0add84-1da1-497d-b34d-583796c8f1f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:32.721627 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0add84-1da1-497d-b34d-583796c8f1f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:32.721911 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0add84-1da1-497d-b34d-583796c8f1f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:32.730814 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0add84-1da1-497d-b34d-583796c8f1f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:32.732287 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0add84-1da1-497d-b34d-583796c8f1f4 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:32.741599 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1292/2586] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:32.782576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362d96c-f88b-47fb-b0a6-718fed1d7d2b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:32.784019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4362d96c-f88b-47fb-b0a6-718fed1d7d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:32.795821 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4362d96c-f88b-47fb-b0a6-718fed1d7d2b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:32.799399 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ebfbf-07c2-43f7-8b87-15f112fcfa77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:32.800319 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ebfbf-07c2-43f7-8b87-15f112fcfa77 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:32.800605 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ebfbf-07c2-43f7-8b87-15f112fcfa77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:32.800883 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ebfbf-07c2-43f7-8b87-15f112fcfa77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:32.858926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ebfbf-07c2-43f7-8b87-15f112fcfa77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:32.860447 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ebfbf-07c2-43f7-8b87-15f112fcfa77 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:32.870121 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1295/2587] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:36.751614 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0add84-1da1-497d-b34d-583796c8f1f4 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:36.755893 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:36.815554 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0add84-1da1-497d-b34d-583796c8f1f4 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:36.820090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33dea0-3213-4c27-ae57-87efe7adcb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:36.821017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33dea0-3213-4c27-ae57-87efe7adcb6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:36.821299 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33dea0-3213-4c27-ae57-87efe7adcb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:36.821576 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33dea0-3213-4c27-ae57-87efe7adcb6e admin admin] PATH_INFO: `/v3/users/8917b39ce5ab4e3ea2bfb846a1e361e9` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:36.822743 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33dea0-3213-4c27-ae57-87efe7adcb6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8917b39ce5ab4e3ea2bfb846a1e361e9)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:36.824698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33dea0-3213-4c27-ae57-87efe7adcb6e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:36.873990 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5b33dea0-3213-4c27-ae57-87efe7adcb6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8917b39ce5ab4e3ea2bfb846a1e361e9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:36.881351 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5b33dea0-3213-4c27-ae57-87efe7adcb6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8917b39ce5ab4e3ea2bfb846a1e361e9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:36.888432 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5b33dea0-3213-4c27-ae57-87efe7adcb6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8917b39ce5ab4e3ea2bfb846a1e361e9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:36.889072 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5b33dea0-3213-4c27-ae57-87efe7adcb6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8917b39ce5ab4e3ea2bfb846a1e361e9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:36.901190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-5b33dea0-3213-4c27-ae57-87efe7adcb6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8917b39ce5ab4e3ea2bfb846a1e361e9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:36.907021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1293/2588] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:36 2022] DELETE /identity/v3/users/8917b39ce5ab4e3ea2bfb846a1e361e9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:36.911584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7760c00-2026-4548-92e8-82b8c0e658f8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:36.912093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7760c00-2026-4548-92e8-82b8c0e658f8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:36.912203 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7760c00-2026-4548-92e8-82b8c0e658f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:36.929745 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f7760c00-2026-4548-92e8-82b8c0e658f8 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:36.931582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f7760c00-2026-4548-92e8-82b8c0e658f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:36.996487 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1296/2589] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:37.006758 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33dea0-3213-4c27-ae57-87efe7adcb6e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:37.008270 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5b33dea0-3213-4c27-ae57-87efe7adcb6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:37.065644 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33dea0-3213-4c27-ae57-87efe7adcb6e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:37.069324 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be9776-d5f8-4f7c-a59a-41839296e5a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:37.069970 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be9776-d5f8-4f7c-a59a-41839296e5a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:37.070261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be9776-d5f8-4f7c-a59a-41839296e5a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:37.070547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be9776-d5f8-4f7c-a59a-41839296e5a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:37.079621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be9776-d5f8-4f7c-a59a-41839296e5a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:37.081096 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be9776-d5f8-4f7c-a59a-41839296e5a5 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:37.091012 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1294/2590] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:20:37.357097 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7760c00-2026-4548-92e8-82b8c0e658f8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:37.358516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f7760c00-2026-4548-92e8-82b8c0e658f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:37.370221 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7760c00-2026-4548-92e8-82b8c0e658f8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:37.374779 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6eed0-bed3-40a9-b0b0-b0b8a47f49c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:37.375637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6eed0-bed3-40a9-b0b0-b0b8a47f49c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:37.375923 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6eed0-bed3-40a9-b0b0-b0b8a47f49c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:37.376259 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6eed0-bed3-40a9-b0b0-b0b8a47f49c3 admin admin] PATH_INFO: `/v3/projects/d937813020b04a3eb724e373664872c3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:37.377434 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6eed0-bed3-40a9-b0b0-b0b8a47f49c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d937813020b04a3eb724e373664872c3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:37.379348 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6eed0-bed3-40a9-b0b0-b0b8a47f49c3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:37.428471 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b2c6eed0-bed3-40a9-b0b0-b0b8a47f49c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd937813020b04a3eb724e373664872c3', 'request_id': 'req-b2c6eed0-bed3-40a9-b0b0-b0b8a47f49c3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:37.435326 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b2c6eed0-bed3-40a9-b0b0-b0b8a47f49c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd937813020b04a3eb724e373664872c3', 'request_id': 'req-b2c6eed0-bed3-40a9-b0b0-b0b8a47f49c3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:37.442588 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b2c6eed0-bed3-40a9-b0b0-b0b8a47f49c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd937813020b04a3eb724e373664872c3', 'request_id': 'req-b2c6eed0-bed3-40a9-b0b0-b0b8a47f49c3'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:37.450247 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:37.457357 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b2c6eed0-bed3-40a9-b0b0-b0b8a47f49c3 admin admin] The token cache is being invalidate because project d937813020b04a3eb724e373664872c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:37.457656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b2c6eed0-bed3-40a9-b0b0-b0b8a47f49c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:37.458717 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b2c6eed0-bed3-40a9-b0b0-b0b8a47f49c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:37.460709 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1297/2591] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:37 2022] DELETE /identity/v3/projects/d937813020b04a3eb724e373664872c3 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:37.466865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be9776-d5f8-4f7c-a59a-41839296e5a5 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:37.476583 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be9776-d5f8-4f7c-a59a-41839296e5a5 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:37.496279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:37.536308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19236a72-fb52-4597-8160-8eaf862a339e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:37.537181 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19236a72-fb52-4597-8160-8eaf862a339e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:37.537464 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19236a72-fb52-4597-8160-8eaf862a339e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:37.537742 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19236a72-fb52-4597-8160-8eaf862a339e admin admin] PATH_INFO: `/v3/users/ed4cf2446adb4b1db35266433cfdcb22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:37.538922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19236a72-fb52-4597-8160-8eaf862a339e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed4cf2446adb4b1db35266433cfdcb22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:37.540896 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19236a72-fb52-4597-8160-8eaf862a339e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:37.584736 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-19236a72-fb52-4597-8160-8eaf862a339e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed4cf2446adb4b1db35266433cfdcb22'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:37.592351 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-19236a72-fb52-4597-8160-8eaf862a339e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed4cf2446adb4b1db35266433cfdcb22'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:37.599055 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-19236a72-fb52-4597-8160-8eaf862a339e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed4cf2446adb4b1db35266433cfdcb22'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:37.599669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-19236a72-fb52-4597-8160-8eaf862a339e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed4cf2446adb4b1db35266433cfdcb22'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:37.611235 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-19236a72-fb52-4597-8160-8eaf862a339e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed4cf2446adb4b1db35266433cfdcb22'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:37.617008 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1295/2592] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:37 2022] DELETE /identity/v3/users/ed4cf2446adb4b1db35266433cfdcb22 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:37.886391 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6eed0-bed3-40a9-b0b0-b0b8a47f49c3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:37.888114 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b2c6eed0-bed3-40a9-b0b0-b0b8a47f49c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:37.902705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6eed0-bed3-40a9-b0b0-b0b8a47f49c3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:37.964466 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83eb80-5982-409a-832f-89ddf5746f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:37.965280 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83eb80-5982-409a-832f-89ddf5746f01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:37.965568 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83eb80-5982-409a-832f-89ddf5746f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:37.965849 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83eb80-5982-409a-832f-89ddf5746f01 admin admin] PATH_INFO: `/v3/projects/a857efebf62b40d187170bbe6f2ca149` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:37.967027 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83eb80-5982-409a-832f-89ddf5746f01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a857efebf62b40d187170bbe6f2ca149)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:37.968967 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83eb80-5982-409a-832f-89ddf5746f01 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:38.014811 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1a83eb80-5982-409a-832f-89ddf5746f01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a857efebf62b40d187170bbe6f2ca149', 'request_id': 'req-1a83eb80-5982-409a-832f-89ddf5746f01'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:38.020784 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1a83eb80-5982-409a-832f-89ddf5746f01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a857efebf62b40d187170bbe6f2ca149', 'request_id': 'req-1a83eb80-5982-409a-832f-89ddf5746f01'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:38.027788 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1a83eb80-5982-409a-832f-89ddf5746f01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a857efebf62b40d187170bbe6f2ca149', 'request_id': 'req-1a83eb80-5982-409a-832f-89ddf5746f01'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:38.038830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1a83eb80-5982-409a-832f-89ddf5746f01 admin admin] The token cache is being invalidate because project a857efebf62b40d187170bbe6f2ca149 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:38.039120 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1a83eb80-5982-409a-832f-89ddf5746f01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:38.039738 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1a83eb80-5982-409a-832f-89ddf5746f01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:38.041278 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1298/2593] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:37 2022] DELETE /identity/v3/projects/a857efebf62b40d187170bbe6f2ca149 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:38.045259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19236a72-fb52-4597-8160-8eaf862a339e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:38.046719 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-19236a72-fb52-4597-8160-8eaf862a339e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:38.060517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19236a72-fb52-4597-8160-8eaf862a339e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:38.115400 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66a089-e12b-403b-80eb-9a9221f32d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:38.116303 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66a089-e12b-403b-80eb-9a9221f32d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:38.116597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66a089-e12b-403b-80eb-9a9221f32d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:38.116911 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66a089-e12b-403b-80eb-9a9221f32d69 admin admin] PATH_INFO: `/v3/users/2f7a14fc635c40ea8b2297c33ea19335` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:38.118694 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed66a089-e12b-403b-80eb-9a9221f32d69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f7a14fc635c40ea8b2297c33ea19335)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:38.120649 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed66a089-e12b-403b-80eb-9a9221f32d69 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:38.163591 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:38.165900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ed66a089-e12b-403b-80eb-9a9221f32d69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f7a14fc635c40ea8b2297c33ea19335'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:38.173495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ed66a089-e12b-403b-80eb-9a9221f32d69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f7a14fc635c40ea8b2297c33ea19335'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:38.181519 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ed66a089-e12b-403b-80eb-9a9221f32d69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f7a14fc635c40ea8b2297c33ea19335'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:38.182149 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ed66a089-e12b-403b-80eb-9a9221f32d69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f7a14fc635c40ea8b2297c33ea19335'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:38.193863 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ed66a089-e12b-403b-80eb-9a9221f32d69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f7a14fc635c40ea8b2297c33ea19335'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:38.199734 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1296/2594] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:38 2022] DELETE /identity/v3/users/2f7a14fc635c40ea8b2297c33ea19335 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:20:38.462979 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83eb80-5982-409a-832f-89ddf5746f01 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:38.464446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1a83eb80-5982-409a-832f-89ddf5746f01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:38.477962 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83eb80-5982-409a-832f-89ddf5746f01 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:38.535115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389e900-6bbe-4237-92c8-c02fc459c36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:38.536011 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389e900-6bbe-4237-92c8-c02fc459c36d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:38.536297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389e900-6bbe-4237-92c8-c02fc459c36d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:38.536574 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389e900-6bbe-4237-92c8-c02fc459c36d admin admin] PATH_INFO: `/v3/projects/af86cae93be44cb1aa17b096cf369723` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:38.537733 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389e900-6bbe-4237-92c8-c02fc459c36d admin admin] RBAC: Authorizing `identity:delete_project(project_id=af86cae93be44cb1aa17b096cf369723)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:38.540337 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389e900-6bbe-4237-92c8-c02fc459c36d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:38.575961 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:38.591737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3389e900-6bbe-4237-92c8-c02fc459c36d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af86cae93be44cb1aa17b096cf369723', 'request_id': 'req-3389e900-6bbe-4237-92c8-c02fc459c36d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:38.597854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3389e900-6bbe-4237-92c8-c02fc459c36d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af86cae93be44cb1aa17b096cf369723', 'request_id': 'req-3389e900-6bbe-4237-92c8-c02fc459c36d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:38.605753 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3389e900-6bbe-4237-92c8-c02fc459c36d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af86cae93be44cb1aa17b096cf369723', 'request_id': 'req-3389e900-6bbe-4237-92c8-c02fc459c36d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:38.616291 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3389e900-6bbe-4237-92c8-c02fc459c36d admin admin] The token cache is being invalidate because project af86cae93be44cb1aa17b096cf369723 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:38.616584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3389e900-6bbe-4237-92c8-c02fc459c36d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:38.617218 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3389e900-6bbe-4237-92c8-c02fc459c36d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:38.618838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1299/2595] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:38 2022] DELETE /identity/v3/projects/af86cae93be44cb1aa17b096cf369723 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:38.624888 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a193b-22a5-46c6-bec7-21dc4de327f0 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:38.625192 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a193b-22a5-46c6-bec7-21dc4de327f0 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:38.625483 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a193b-22a5-46c6-bec7-21dc4de327f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:38.643463 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-aa4a193b-22a5-46c6-bec7-21dc4de327f0 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:38.644706 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-aa4a193b-22a5-46c6-bec7-21dc4de327f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:38.703667 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1297/2596] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:38.712505 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b12e39-cd40-45a6-a275-c957ff2637c0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:38.712811 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b12e39-cd40-45a6-a275-c957ff2637c0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:38.713099 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b12e39-cd40-45a6-a275-c957ff2637c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:38.731782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-24b12e39-cd40-45a6-a275-c957ff2637c0 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:38.733005 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-24b12e39-cd40-45a6-a275-c957ff2637c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:38.741191 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1300/2597] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:38.745165 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a193b-22a5-46c6-bec7-21dc4de327f0 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:38.754859 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a193b-22a5-46c6-bec7-21dc4de327f0 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:38.759505 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62effe81-feb6-45c5-9797-75086a60e0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:38.760193 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62effe81-feb6-45c5-9797-75086a60e0b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:38.760484 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62effe81-feb6-45c5-9797-75086a60e0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:38.760768 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62effe81-feb6-45c5-9797-75086a60e0b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:38.761500 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62effe81-feb6-45c5-9797-75086a60e0b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:20:38.761810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62effe81-feb6-45c5-9797-75086a60e0b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:38.763407 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62effe81-feb6-45c5-9797-75086a60e0b7 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:38.775464 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1298/2598] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:20:38.779729 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b12e39-cd40-45a6-a275-c957ff2637c0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:38.788956 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b12e39-cd40-45a6-a275-c957ff2637c0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:38.794528 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018d88b-9862-43ac-9870-23589a3c30cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:38.795195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018d88b-9862-43ac-9870-23589a3c30cd admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:38.795486 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018d88b-9862-43ac-9870-23589a3c30cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:38.795771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018d88b-9862-43ac-9870-23589a3c30cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:38.796512 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018d88b-9862-43ac-9870-23589a3c30cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:38.798435 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018d88b-9862-43ac-9870-23589a3c30cd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:38.828643 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1301/2599] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:38 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:38.829757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62effe81-feb6-45c5-9797-75086a60e0b7 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:38.840796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62effe81-feb6-45c5-9797-75086a60e0b7 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:38.896779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b31d57-92e9-41b6-81ab-ace6e0464109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:38.897457 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b31d57-92e9-41b6-81ab-ace6e0464109 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:38.897745 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b31d57-92e9-41b6-81ab-ace6e0464109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:38.898051 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b31d57-92e9-41b6-81ab-ace6e0464109 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:38.898817 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b31d57-92e9-41b6-81ab-ace6e0464109 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:38.901267 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b31d57-92e9-41b6-81ab-ace6e0464109 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:38.926037 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1299/2600] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:38 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:38.927508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018d88b-9862-43ac-9870-23589a3c30cd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:38.938238 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018d88b-9862-43ac-9870-23589a3c30cd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:38.943558 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82b45f-81a1-49f2-9d20-2d7ef27789a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:38.944498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82b45f-81a1-49f2-9d20-2d7ef27789a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:38.944793 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82b45f-81a1-49f2-9d20-2d7ef27789a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:38.945075 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82b45f-81a1-49f2-9d20-2d7ef27789a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:38.946338 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82b45f-81a1-49f2-9d20-2d7ef27789a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:38.947992 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82b45f-81a1-49f2-9d20-2d7ef27789a5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:38.959824 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1302/2601] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:38.964616 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b31d57-92e9-41b6-81ab-ace6e0464109 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:38.975208 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b31d57-92e9-41b6-81ab-ace6e0464109 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:38.980080 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a649a-4a2b-42e1-a605-3e486d680861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:38.980803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a649a-4a2b-42e1-a605-3e486d680861 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:38.981096 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a649a-4a2b-42e1-a605-3e486d680861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:38.981393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a649a-4a2b-42e1-a605-3e486d680861 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:38.982325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558a649a-4a2b-42e1-a605-3e486d680861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:38.983913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558a649a-4a2b-42e1-a605-3e486d680861 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:38.995186 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1300/2602] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:38.998666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82b45f-81a1-49f2-9d20-2d7ef27789a5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:39.009170 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82b45f-81a1-49f2-9d20-2d7ef27789a5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:39.013749 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c9f91-381f-49f7-b008-5b9eaf7de8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:39.014615 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c9f91-381f-49f7-b008-5b9eaf7de8d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:39.014898 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c9f91-381f-49f7-b008-5b9eaf7de8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:39.015177 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c9f91-381f-49f7-b008-5b9eaf7de8d6 admin admin] PATH_INFO: `/v3/projects/5bc0f8e973894e53afadacd583156b3b/users/ffda5e9aeda040258e04dc4c4de1b845/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:39.029964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c9f91-381f-49f7-b008-5b9eaf7de8d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bc0f8e973894e53afadacd583156b3b, user_id=ffda5e9aeda040258e04dc4c4de1b845, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:39.032679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c9f91-381f-49f7-b008-5b9eaf7de8d6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:39.046303 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1303/2603] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:38 2022] PUT /identity/v3/projects/5bc0f8e973894e53afadacd583156b3b/users/ffda5e9aeda040258e04dc4c4de1b845/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:39.058208 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c232d-e2f2-4c4e-966d-65f8c51648f8 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:39.058689 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c232d-e2f2-4c4e-966d-65f8c51648f8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:39.058812 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c232d-e2f2-4c4e-966d-65f8c51648f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:39.085919 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-bd3c232d-e2f2-4c4e-966d-65f8c51648f8 None None] MFA Rules not processed for user `ffda5e9aeda040258e04dc4c4de1b845`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:39.087938 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bd3c232d-e2f2-4c4e-966d-65f8c51648f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:39.105792 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:39.155376 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1301/2604] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:39.159903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c9f91-381f-49f7-b008-5b9eaf7de8d6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:39.223177 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c9f91-381f-49f7-b008-5b9eaf7de8d6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:39.227861 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da118ea5-6969-4a9e-ae8d-e9dfb2662694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:39.228545 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da118ea5-6969-4a9e-ae8d-e9dfb2662694 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:39.228835 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da118ea5-6969-4a9e-ae8d-e9dfb2662694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:39.229115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da118ea5-6969-4a9e-ae8d-e9dfb2662694 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:39.229818 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da118ea5-6969-4a9e-ae8d-e9dfb2662694 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:39.231710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da118ea5-6969-4a9e-ae8d-e9dfb2662694 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:39.256437 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1304/2605] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:39 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:39.260981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c232d-e2f2-4c4e-966d-65f8c51648f8 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:39.271584 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c232d-e2f2-4c4e-966d-65f8c51648f8 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:39.328394 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee85daa-cd48-4d23-b7bc-c5d451e15d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:39.329070 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee85daa-cd48-4d23-b7bc-c5d451e15d94 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:39.329361 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee85daa-cd48-4d23-b7bc-c5d451e15d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:39.329673 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee85daa-cd48-4d23-b7bc-c5d451e15d94 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:39.330371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee85daa-cd48-4d23-b7bc-c5d451e15d94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:39.332339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee85daa-cd48-4d23-b7bc-c5d451e15d94 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:39.352087 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1302/2606] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:39 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:39.356111 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da118ea5-6969-4a9e-ae8d-e9dfb2662694 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:39.366239 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da118ea5-6969-4a9e-ae8d-e9dfb2662694 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:39.371382 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dade1e8-761e-4b20-b10a-c82429f70dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:39.372065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dade1e8-761e-4b20-b10a-c82429f70dba admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:39.372354 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dade1e8-761e-4b20-b10a-c82429f70dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:39.372634 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dade1e8-761e-4b20-b10a-c82429f70dba admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:39.373353 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dade1e8-761e-4b20-b10a-c82429f70dba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:39.374939 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dade1e8-761e-4b20-b10a-c82429f70dba admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:39.385856 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1305/2607] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:39.389848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee85daa-cd48-4d23-b7bc-c5d451e15d94 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:39.400823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee85daa-cd48-4d23-b7bc-c5d451e15d94 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:39.405475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e0884-831e-4bf6-b2fd-9e43c625ff5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:39.406417 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e0884-831e-4bf6-b2fd-9e43c625ff5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:39.406767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e0884-831e-4bf6-b2fd-9e43c625ff5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:39.407083 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e0884-831e-4bf6-b2fd-9e43c625ff5e admin admin] PATH_INFO: `/v3/projects/42302f0e05a948b68279e8b2b86a48f4/users/7175fe4f67f3411a8486a76f9ee61dbb/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:39.422777 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3e0884-831e-4bf6-b2fd-9e43c625ff5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=42302f0e05a948b68279e8b2b86a48f4, user_id=7175fe4f67f3411a8486a76f9ee61dbb, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:39.425783 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3e0884-831e-4bf6-b2fd-9e43c625ff5e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:39.439261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1303/2608] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:39 2022] PUT /identity/v3/projects/42302f0e05a948b68279e8b2b86a48f4/users/7175fe4f67f3411a8486a76f9ee61dbb/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:39.443124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dade1e8-761e-4b20-b10a-c82429f70dba admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:39.453716 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dade1e8-761e-4b20-b10a-c82429f70dba admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:39.478685 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:39.511769 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d25dc6-9559-4a3b-ad5f-24eef2b43e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:39.512400 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d25dc6-9559-4a3b-ad5f-24eef2b43e52 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:39.512687 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d25dc6-9559-4a3b-ad5f-24eef2b43e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:39.512964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d25dc6-9559-4a3b-ad5f-24eef2b43e52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:39.513685 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d25dc6-9559-4a3b-ad5f-24eef2b43e52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:39.515281 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d25dc6-9559-4a3b-ad5f-24eef2b43e52 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:39.525101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1306/2609] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:39.529394 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e0884-831e-4bf6-b2fd-9e43c625ff5e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:39.539625 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e0884-831e-4bf6-b2fd-9e43c625ff5e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:39.544284 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634eb55-3634-4a40-a34d-81242f950a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:39.545167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634eb55-3634-4a40-a34d-81242f950a99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:39.545457 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634eb55-3634-4a40-a34d-81242f950a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:39.545746 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634eb55-3634-4a40-a34d-81242f950a99 admin admin] PATH_INFO: `/v3/projects/42302f0e05a948b68279e8b2b86a48f4/users/7175fe4f67f3411a8486a76f9ee61dbb/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:39.547731 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634eb55-3634-4a40-a34d-81242f950a99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42302f0e05a948b68279e8b2b86a48f4, user_id=7175fe4f67f3411a8486a76f9ee61dbb, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:39.550384 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634eb55-3634-4a40-a34d-81242f950a99 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:39.561214 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1304/2610] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:39 2022] PUT /identity/v3/projects/42302f0e05a948b68279e8b2b86a48f4/users/7175fe4f67f3411a8486a76f9ee61dbb/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:39.572595 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6373d35-d47f-46d2-b2df-e965d01bf094 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:39.572893 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6373d35-d47f-46d2-b2df-e965d01bf094 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:39.573185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6373d35-d47f-46d2-b2df-e965d01bf094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:39.598607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f6373d35-d47f-46d2-b2df-e965d01bf094 None None] MFA Rules not processed for user `7175fe4f67f3411a8486a76f9ee61dbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:39.599826 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f6373d35-d47f-46d2-b2df-e965d01bf094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:39.666476 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1307/2611] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:39 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:20:39.678342 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634eb55-3634-4a40-a34d-81242f950a99 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:39.680254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e634eb55-3634-4a40-a34d-81242f950a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:39.737032 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634eb55-3634-4a40-a34d-81242f950a99 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:39.740775 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a37fd5-7669-4719-9b02-ce2193935e25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:39.741496 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a37fd5-7669-4719-9b02-ce2193935e25 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:39.741843 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a37fd5-7669-4719-9b02-ce2193935e25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:39.742160 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a37fd5-7669-4719-9b02-ce2193935e25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:39.752346 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a37fd5-7669-4719-9b02-ce2193935e25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:39.753340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a37fd5-7669-4719-9b02-ce2193935e25 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:39.762854 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1305/2612] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:39 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:39.841852 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6373d35-d47f-46d2-b2df-e965d01bf094 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:39.851411 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6373d35-d47f-46d2-b2df-e965d01bf094 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:39.855518 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be20f76-c7c6-41ed-9dd9-a45cbaa4d019 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:39.856187 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be20f76-c7c6-41ed-9dd9-a45cbaa4d019 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:39.856498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be20f76-c7c6-41ed-9dd9-a45cbaa4d019 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:39.856759 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be20f76-c7c6-41ed-9dd9-a45cbaa4d019 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:39.913126 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be20f76-c7c6-41ed-9dd9-a45cbaa4d019 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:39.915376 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be20f76-c7c6-41ed-9dd9-a45cbaa4d019 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:39.925219 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1308/2613] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:39 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:43.749861 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a37fd5-7669-4719-9b02-ce2193935e25 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:43.751691 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-39a37fd5-7669-4719-9b02-ce2193935e25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:43.763645 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a37fd5-7669-4719-9b02-ce2193935e25 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:43.767718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9aacd3-035a-4c87-ab3b-649159f3bd96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:43.768405 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9aacd3-035a-4c87-ab3b-649159f3bd96 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:43.768696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9aacd3-035a-4c87-ab3b-649159f3bd96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:43.769214 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9aacd3-035a-4c87-ab3b-649159f3bd96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:43.771001 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ab9aacd3-035a-4c87-ab3b-649159f3bd96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:43.784859 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:43.838624 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9aacd3-035a-4c87-ab3b-649159f3bd96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:43.842005 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9aacd3-035a-4c87-ab3b-649159f3bd96 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:43.852467 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1306/2614] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:44.610244 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be20f76-c7c6-41ed-9dd9-a45cbaa4d019 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:44.657771 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:44.672948 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be20f76-c7c6-41ed-9dd9-a45cbaa4d019 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:44.677871 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b7631-056a-48d2-b494-86370919c21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:44.678743 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b7631-056a-48d2-b494-86370919c21f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:44.679064 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b7631-056a-48d2-b494-86370919c21f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:44.679348 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b7631-056a-48d2-b494-86370919c21f admin admin] PATH_INFO: `/v3/users/ffda5e9aeda040258e04dc4c4de1b845` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:44.680561 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3b7631-056a-48d2-b494-86370919c21f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffda5e9aeda040258e04dc4c4de1b845)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:44.682515 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3b7631-056a-48d2-b494-86370919c21f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:44.727517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7a3b7631-056a-48d2-b494-86370919c21f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffda5e9aeda040258e04dc4c4de1b845'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:44.735065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7a3b7631-056a-48d2-b494-86370919c21f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffda5e9aeda040258e04dc4c4de1b845'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:44.742363 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7a3b7631-056a-48d2-b494-86370919c21f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffda5e9aeda040258e04dc4c4de1b845'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:44.743031 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7a3b7631-056a-48d2-b494-86370919c21f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffda5e9aeda040258e04dc4c4de1b845'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:44.754608 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7a3b7631-056a-48d2-b494-86370919c21f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffda5e9aeda040258e04dc4c4de1b845'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:44.760467 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1309/2615] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:44 2022] DELETE /identity/v3/users/ffda5e9aeda040258e04dc4c4de1b845 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:44.765157 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329be767-09de-4ead-8779-7ce51e7bf11e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:44.765615 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329be767-09de-4ead-8779-7ce51e7bf11e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:44.765729 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329be767-09de-4ead-8779-7ce51e7bf11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:44.783700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-329be767-09de-4ead-8779-7ce51e7bf11e None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:44.784938 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-329be767-09de-4ead-8779-7ce51e7bf11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:44.843571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1307/2616] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:44.853400 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b7631-056a-48d2-b494-86370919c21f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:44.854816 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7a3b7631-056a-48d2-b494-86370919c21f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:44.911003 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b7631-056a-48d2-b494-86370919c21f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:44.914658 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ef446-2bc4-4065-a37a-1711ebfcaa76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:44.915347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ef446-2bc4-4065-a37a-1711ebfcaa76 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:44.915641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ef446-2bc4-4065-a37a-1711ebfcaa76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:44.915925 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ef446-2bc4-4065-a37a-1711ebfcaa76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:44.926026 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ef446-2bc4-4065-a37a-1711ebfcaa76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:44.927820 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ef446-2bc4-4065-a37a-1711ebfcaa76 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:44.937902 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1310/2617] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:45.214401 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329be767-09de-4ead-8779-7ce51e7bf11e None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:45.215861 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-329be767-09de-4ead-8779-7ce51e7bf11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:45.229129 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329be767-09de-4ead-8779-7ce51e7bf11e None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:45.233766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85eb53-e6b3-47f5-a703-b1fe267ae9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:45.234631 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85eb53-e6b3-47f5-a703-b1fe267ae9a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:45.235239 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85eb53-e6b3-47f5-a703-b1fe267ae9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:45.235871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85eb53-e6b3-47f5-a703-b1fe267ae9a8 admin admin] PATH_INFO: `/v3/projects/5bc0f8e973894e53afadacd583156b3b` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:45.237277 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85eb53-e6b3-47f5-a703-b1fe267ae9a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bc0f8e973894e53afadacd583156b3b)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:45.239213 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85eb53-e6b3-47f5-a703-b1fe267ae9a8 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:45.286150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3a85eb53-e6b3-47f5-a703-b1fe267ae9a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bc0f8e973894e53afadacd583156b3b', 'request_id': 'req-3a85eb53-e6b3-47f5-a703-b1fe267ae9a8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.291536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3a85eb53-e6b3-47f5-a703-b1fe267ae9a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bc0f8e973894e53afadacd583156b3b', 'request_id': 'req-3a85eb53-e6b3-47f5-a703-b1fe267ae9a8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.298735 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3a85eb53-e6b3-47f5-a703-b1fe267ae9a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bc0f8e973894e53afadacd583156b3b', 'request_id': 'req-3a85eb53-e6b3-47f5-a703-b1fe267ae9a8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.310451 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3a85eb53-e6b3-47f5-a703-b1fe267ae9a8 admin admin] The token cache is being invalidate because project 5bc0f8e973894e53afadacd583156b3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:45.310743 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3a85eb53-e6b3-47f5-a703-b1fe267ae9a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.311372 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-3a85eb53-e6b3-47f5-a703-b1fe267ae9a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.312938 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1308/2618] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:45 2022] DELETE /identity/v3/projects/5bc0f8e973894e53afadacd583156b3b => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:45.316923 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ef446-2bc4-4065-a37a-1711ebfcaa76 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:45.327112 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ef446-2bc4-4065-a37a-1711ebfcaa76 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:45.383026 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5a788-449a-4f4c-a759-1914df33f427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:45.384589 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5a788-449a-4f4c-a759-1914df33f427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:45.384880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5a788-449a-4f4c-a759-1914df33f427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:45.385161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5a788-449a-4f4c-a759-1914df33f427 admin admin] PATH_INFO: `/v3/users/7175fe4f67f3411a8486a76f9ee61dbb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:45.386347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a5a788-449a-4f4c-a759-1914df33f427 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7175fe4f67f3411a8486a76f9ee61dbb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:45.388344 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a5a788-449a-4f4c-a759-1914df33f427 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:45.401604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:45.434927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-01a5a788-449a-4f4c-a759-1914df33f427 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7175fe4f67f3411a8486a76f9ee61dbb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.442993 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-01a5a788-449a-4f4c-a759-1914df33f427 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7175fe4f67f3411a8486a76f9ee61dbb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.450540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-01a5a788-449a-4f4c-a759-1914df33f427 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7175fe4f67f3411a8486a76f9ee61dbb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.451229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-01a5a788-449a-4f4c-a759-1914df33f427 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7175fe4f67f3411a8486a76f9ee61dbb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.463521 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-01a5a788-449a-4f4c-a759-1914df33f427 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7175fe4f67f3411a8486a76f9ee61dbb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.469499 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1311/2619] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:45 2022] DELETE /identity/v3/users/7175fe4f67f3411a8486a76f9ee61dbb => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:45.745525 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85eb53-e6b3-47f5-a703-b1fe267ae9a8 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:45.746999 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3a85eb53-e6b3-47f5-a703-b1fe267ae9a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:45.761779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85eb53-e6b3-47f5-a703-b1fe267ae9a8 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:45.795819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:45.822348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e68c8-744c-4eaf-b31b-6e2f37ed49bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:45.823141 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e68c8-744c-4eaf-b31b-6e2f37ed49bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:45.823437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e68c8-744c-4eaf-b31b-6e2f37ed49bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:45.823711 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e68c8-744c-4eaf-b31b-6e2f37ed49bf admin admin] PATH_INFO: `/v3/projects/42302f0e05a948b68279e8b2b86a48f4` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:45.824932 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e68c8-744c-4eaf-b31b-6e2f37ed49bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=42302f0e05a948b68279e8b2b86a48f4)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:45.826826 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e68c8-744c-4eaf-b31b-6e2f37ed49bf admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:45.873241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4a0e68c8-744c-4eaf-b31b-6e2f37ed49bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42302f0e05a948b68279e8b2b86a48f4', 'request_id': 'req-4a0e68c8-744c-4eaf-b31b-6e2f37ed49bf'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.878739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4a0e68c8-744c-4eaf-b31b-6e2f37ed49bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42302f0e05a948b68279e8b2b86a48f4', 'request_id': 'req-4a0e68c8-744c-4eaf-b31b-6e2f37ed49bf'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.885664 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4a0e68c8-744c-4eaf-b31b-6e2f37ed49bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42302f0e05a948b68279e8b2b86a48f4', 'request_id': 'req-4a0e68c8-744c-4eaf-b31b-6e2f37ed49bf'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.896445 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4a0e68c8-744c-4eaf-b31b-6e2f37ed49bf admin admin] The token cache is being invalidate because project 42302f0e05a948b68279e8b2b86a48f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:45.896737 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4a0e68c8-744c-4eaf-b31b-6e2f37ed49bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.897362 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4a0e68c8-744c-4eaf-b31b-6e2f37ed49bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.898973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1309/2620] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:45 2022] DELETE /identity/v3/projects/42302f0e05a948b68279e8b2b86a48f4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:45.903873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbbd0c-b291-489a-96d9-9c52ef551038 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:45.904360 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbbd0c-b291-489a-96d9-9c52ef551038 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:45.904557 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbbd0c-b291-489a-96d9-9c52ef551038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:45.922216 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b8fbbd0c-b291-489a-96d9-9c52ef551038 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:45.923409 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b8fbbd0c-b291-489a-96d9-9c52ef551038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:45.983110 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1312/2621] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:20:45.990963 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688ab5b-af3c-4595-8a56-021c701fd589 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:45.991261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688ab5b-af3c-4595-8a56-021c701fd589 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:45.991544 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688ab5b-af3c-4595-8a56-021c701fd589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.009491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-1688ab5b-af3c-4595-8a56-021c701fd589 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:46.010700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1688ab5b-af3c-4595-8a56-021c701fd589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:46.018561 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1310/2622] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:46.022943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbbd0c-b291-489a-96d9-9c52ef551038 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:46.033229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbbd0c-b291-489a-96d9-9c52ef551038 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:46.037805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5f31a-5bd1-4dee-a229-ef645afd831a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:46.038470 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5f31a-5bd1-4dee-a229-ef645afd831a admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.038762 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5f31a-5bd1-4dee-a229-ef645afd831a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.039045 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5f31a-5bd1-4dee-a229-ef645afd831a admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.039783 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5f31a-5bd1-4dee-a229-ef645afd831a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:20:46.040768 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5f31a-5bd1-4dee-a229-ef645afd831a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:46.042406 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5f31a-5bd1-4dee-a229-ef645afd831a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:46.054647 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1313/2623] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:20:46.059278 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688ab5b-af3c-4595-8a56-021c701fd589 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:46.069443 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688ab5b-af3c-4595-8a56-021c701fd589 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:46.075304 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a72fd-4ae7-46bd-b694-a515b463b2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:46.076057 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a72fd-4ae7-46bd-b694-a515b463b2c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.076442 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a72fd-4ae7-46bd-b694-a515b463b2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.076828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a72fd-4ae7-46bd-b694-a515b463b2c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.077637 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a72fd-4ae7-46bd-b694-a515b463b2c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:46.079988 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a72fd-4ae7-46bd-b694-a515b463b2c4 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:46.107315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1311/2624] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:46 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:46.111302 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5f31a-5bd1-4dee-a229-ef645afd831a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:46.121658 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5f31a-5bd1-4dee-a229-ef645afd831a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:46.175403 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4fb4ed-5eb0-4b53-84d5-3fadafc81b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:46.176091 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fb4ed-5eb0-4b53-84d5-3fadafc81b7e admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.176643 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fb4ed-5eb0-4b53-84d5-3fadafc81b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.176922 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fb4ed-5eb0-4b53-84d5-3fadafc81b7e admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.177611 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4fb4ed-5eb0-4b53-84d5-3fadafc81b7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:46.179539 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4fb4ed-5eb0-4b53-84d5-3fadafc81b7e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:46.199142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1314/2625] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:46 2022] POST /identity/v3/users => generated 293 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:20:46.203166 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a72fd-4ae7-46bd-b694-a515b463b2c4 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:46.213384 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a72fd-4ae7-46bd-b694-a515b463b2c4 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:46.217970 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f29d8-116f-4b62-ad77-55da64a98af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:46.218631 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f29d8-116f-4b62-ad77-55da64a98af2 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.218919 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f29d8-116f-4b62-ad77-55da64a98af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.219201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f29d8-116f-4b62-ad77-55da64a98af2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.219933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f29d8-116f-4b62-ad77-55da64a98af2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:46.221588 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f29d8-116f-4b62-ad77-55da64a98af2 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:46.231882 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1312/2626] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:46.235657 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4fb4ed-5eb0-4b53-84d5-3fadafc81b7e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:46.246365 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4fb4ed-5eb0-4b53-84d5-3fadafc81b7e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:46.251036 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b193e-2cb6-4e58-8115-8e0815dcf22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:46.251692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b193e-2cb6-4e58-8115-8e0815dcf22b admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.252010 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b193e-2cb6-4e58-8115-8e0815dcf22b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.252297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b193e-2cb6-4e58-8115-8e0815dcf22b admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.253015 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b193e-2cb6-4e58-8115-8e0815dcf22b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:46.254603 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b193e-2cb6-4e58-8115-8e0815dcf22b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:46.255854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:46.270154 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1315/2627] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:46.271038 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f29d8-116f-4b62-ad77-55da64a98af2 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:46.281518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f29d8-116f-4b62-ad77-55da64a98af2 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:46.286187 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679dfdc9-243b-4d20-a3ee-e8e5fbb5396b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:46.287061 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679dfdc9-243b-4d20-a3ee-e8e5fbb5396b admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.287350 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679dfdc9-243b-4d20-a3ee-e8e5fbb5396b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.287634 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679dfdc9-243b-4d20-a3ee-e8e5fbb5396b admin admin] PATH_INFO: `/v3/projects/c69e4e50bd3b45178d98cabcb530eb16/users/c550eaa88c6a46ad90f642586d5a2aea/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.303238 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679dfdc9-243b-4d20-a3ee-e8e5fbb5396b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c69e4e50bd3b45178d98cabcb530eb16, user_id=c550eaa88c6a46ad90f642586d5a2aea, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:46.306003 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679dfdc9-243b-4d20-a3ee-e8e5fbb5396b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:46.320654 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1313/2628] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:46 2022] PUT /identity/v3/projects/c69e4e50bd3b45178d98cabcb530eb16/users/c550eaa88c6a46ad90f642586d5a2aea/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:20:46.333057 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02959b73-b26e-4f21-8717-52052b26f679 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.333315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02959b73-b26e-4f21-8717-52052b26f679 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.333569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02959b73-b26e-4f21-8717-52052b26f679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.360468 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-02959b73-b26e-4f21-8717-52052b26f679 None None] MFA Rules not processed for user `c550eaa88c6a46ad90f642586d5a2aea`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:46.361653 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-02959b73-b26e-4f21-8717-52052b26f679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:46.423226 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1316/2629] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:46.434435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679dfdc9-243b-4d20-a3ee-e8e5fbb5396b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:46.436181 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-679dfdc9-243b-4d20-a3ee-e8e5fbb5396b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:46.492174 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679dfdc9-243b-4d20-a3ee-e8e5fbb5396b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:46.495996 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfda764-84f5-4f52-8a96-3489886c7809 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:46.496642 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfda764-84f5-4f52-8a96-3489886c7809 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.496932 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfda764-84f5-4f52-8a96-3489886c7809 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.497215 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfda764-84f5-4f52-8a96-3489886c7809 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.506176 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfda764-84f5-4f52-8a96-3489886c7809 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:46.507619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfda764-84f5-4f52-8a96-3489886c7809 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:46.517131 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1314/2630] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:46.536257 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02959b73-b26e-4f21-8717-52052b26f679 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:46.598838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02959b73-b26e-4f21-8717-52052b26f679 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:46.603475 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdde94-e373-44c2-a915-43dd7060c150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:46.604408 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdde94-e373-44c2-a915-43dd7060c150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.604725 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdde94-e373-44c2-a915-43dd7060c150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.605033 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdde94-e373-44c2-a915-43dd7060c150 admin admin] PATH_INFO: `/v3/users/c550eaa88c6a46ad90f642586d5a2aea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.606354 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fdde94-e373-44c2-a915-43dd7060c150 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c550eaa88c6a46ad90f642586d5a2aea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:46.608383 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fdde94-e373-44c2-a915-43dd7060c150 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:46.654657 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e2fdde94-e373-44c2-a915-43dd7060c150 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c550eaa88c6a46ad90f642586d5a2aea'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:46.662070 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e2fdde94-e373-44c2-a915-43dd7060c150 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c550eaa88c6a46ad90f642586d5a2aea'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:46.669443 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e2fdde94-e373-44c2-a915-43dd7060c150 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c550eaa88c6a46ad90f642586d5a2aea'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:46.670095 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e2fdde94-e373-44c2-a915-43dd7060c150 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c550eaa88c6a46ad90f642586d5a2aea'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:46.682111 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e2fdde94-e373-44c2-a915-43dd7060c150 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c550eaa88c6a46ad90f642586d5a2aea'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:46.688032 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1317/2631] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:46 2022] DELETE /identity/v3/users/c550eaa88c6a46ad90f642586d5a2aea => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:46.692595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a477d5-e44f-4995-9309-19802480ee99 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.692889 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a477d5-e44f-4995-9309-19802480ee99 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.693169 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a477d5-e44f-4995-9309-19802480ee99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.700400 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:46.711490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-36a477d5-e44f-4995-9309-19802480ee99 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:46.713309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-36a477d5-e44f-4995-9309-19802480ee99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:46.772772 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1315/2632] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:46.782362 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdde94-e373-44c2-a915-43dd7060c150 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:46.783828 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e2fdde94-e373-44c2-a915-43dd7060c150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:46.790438 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:46.842863 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdde94-e373-44c2-a915-43dd7060c150 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:46.846589 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0abe2-3042-4c52-9974-98e0e62bd04d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:46.847236 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0abe2-3042-4c52-9974-98e0e62bd04d service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.847529 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0abe2-3042-4c52-9974-98e0e62bd04d service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.847812 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0abe2-3042-4c52-9974-98e0e62bd04d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.857502 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0abe2-3042-4c52-9974-98e0e62bd04d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:46.858860 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0abe2-3042-4c52-9974-98e0e62bd04d service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:46.868761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1318/2633] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:47.149355 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a477d5-e44f-4995-9309-19802480ee99 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.151633 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-36a477d5-e44f-4995-9309-19802480ee99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:47.179017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a477d5-e44f-4995-9309-19802480ee99 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.183916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647562b8-9114-40a7-9d0b-5a0048bc1f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.184855 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647562b8-9114-40a7-9d0b-5a0048bc1f87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.185146 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647562b8-9114-40a7-9d0b-5a0048bc1f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.185427 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647562b8-9114-40a7-9d0b-5a0048bc1f87 admin admin] PATH_INFO: `/v3/projects/c69e4e50bd3b45178d98cabcb530eb16` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.186622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647562b8-9114-40a7-9d0b-5a0048bc1f87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c69e4e50bd3b45178d98cabcb530eb16)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.188584 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647562b8-9114-40a7-9d0b-5a0048bc1f87 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.236299 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-647562b8-9114-40a7-9d0b-5a0048bc1f87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c69e4e50bd3b45178d98cabcb530eb16', 'request_id': 'req-647562b8-9114-40a7-9d0b-5a0048bc1f87'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:47.242388 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-647562b8-9114-40a7-9d0b-5a0048bc1f87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c69e4e50bd3b45178d98cabcb530eb16', 'request_id': 'req-647562b8-9114-40a7-9d0b-5a0048bc1f87'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:47.248868 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-647562b8-9114-40a7-9d0b-5a0048bc1f87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c69e4e50bd3b45178d98cabcb530eb16', 'request_id': 'req-647562b8-9114-40a7-9d0b-5a0048bc1f87'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:47.260334 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-647562b8-9114-40a7-9d0b-5a0048bc1f87 admin admin] The token cache is being invalidate because project c69e4e50bd3b45178d98cabcb530eb16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:47.260620 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-647562b8-9114-40a7-9d0b-5a0048bc1f87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:47.261237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-647562b8-9114-40a7-9d0b-5a0048bc1f87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:47.262786 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1316/2634] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:47 2022] DELETE /identity/v3/projects/c69e4e50bd3b45178d98cabcb530eb16 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:47.267595 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b2d23-9a2c-47b2-a8e9-2f340cf2593f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.268085 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b2d23-9a2c-47b2-a8e9-2f340cf2593f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.268183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b2d23-9a2c-47b2-a8e9-2f340cf2593f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.286183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-df6b2d23-9a2c-47b2-a8e9-2f340cf2593f None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:47.287387 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-df6b2d23-9a2c-47b2-a8e9-2f340cf2593f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:47.345705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1319/2635] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:47.353517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa3226-b94e-4a6b-a2ef-ec06840cd68b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.353814 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa3226-b94e-4a6b-a2ef-ec06840cd68b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.354101 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa3226-b94e-4a6b-a2ef-ec06840cd68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.372548 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-a3fa3226-b94e-4a6b-a2ef-ec06840cd68b None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:47.373672 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-a3fa3226-b94e-4a6b-a2ef-ec06840cd68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:47.381449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1317/2636] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:47.386192 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b2d23-9a2c-47b2-a8e9-2f340cf2593f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.395612 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b2d23-9a2c-47b2-a8e9-2f340cf2593f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.400720 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7c070-e03c-4031-82a7-49b9aedbbac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.401360 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7c070-e03c-4031-82a7-49b9aedbbac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.401643 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7c070-e03c-4031-82a7-49b9aedbbac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.401919 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7c070-e03c-4031-82a7-49b9aedbbac6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.402640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d7c070-e03c-4031-82a7-49b9aedbbac6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:20:47.402944 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d7c070-e03c-4031-82a7-49b9aedbbac6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.404602 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d7c070-e03c-4031-82a7-49b9aedbbac6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.415732 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1320/2637] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:20:47.419834 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa3226-b94e-4a6b-a2ef-ec06840cd68b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.429296 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa3226-b94e-4a6b-a2ef-ec06840cd68b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.433897 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320176dd-c93b-4492-950b-983b352b6c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.434539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320176dd-c93b-4492-950b-983b352b6c09 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.434827 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320176dd-c93b-4492-950b-983b352b6c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.435105 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320176dd-c93b-4492-950b-983b352b6c09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.435792 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320176dd-c93b-4492-950b-983b352b6c09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.437732 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320176dd-c93b-4492-950b-983b352b6c09 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.463366 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1318/2638] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:47 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:47.467260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7c070-e03c-4031-82a7-49b9aedbbac6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.477104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7c070-e03c-4031-82a7-49b9aedbbac6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.534639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818aae4-e583-431d-be88-62df47316a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.535322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818aae4-e583-431d-be88-62df47316a9b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.535613 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818aae4-e583-431d-be88-62df47316a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.535898 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818aae4-e583-431d-be88-62df47316a9b admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.536640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818aae4-e583-431d-be88-62df47316a9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.538605 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818aae4-e583-431d-be88-62df47316a9b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.559127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1321/2639] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:47 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:47.563195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320176dd-c93b-4492-950b-983b352b6c09 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.573991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320176dd-c93b-4492-950b-983b352b6c09 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.578848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6280663-e7d1-44a4-bd5b-6fab04a3bf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.579519 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6280663-e7d1-44a4-bd5b-6fab04a3bf7a admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.579804 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6280663-e7d1-44a4-bd5b-6fab04a3bf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.580138 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6280663-e7d1-44a4-bd5b-6fab04a3bf7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.580896 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6280663-e7d1-44a4-bd5b-6fab04a3bf7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.582508 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6280663-e7d1-44a4-bd5b-6fab04a3bf7a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.592916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1319/2640] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:47.596693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818aae4-e583-431d-be88-62df47316a9b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.606692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818aae4-e583-431d-be88-62df47316a9b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.611297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5437f-1743-43d5-b0a7-2556abdf0716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.612013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5437f-1743-43d5-b0a7-2556abdf0716 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.612307 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5437f-1743-43d5-b0a7-2556abdf0716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.612591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5437f-1743-43d5-b0a7-2556abdf0716 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.613320 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5437f-1743-43d5-b0a7-2556abdf0716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.614934 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5437f-1743-43d5-b0a7-2556abdf0716 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.625987 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1322/2641] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:47.629937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6280663-e7d1-44a4-bd5b-6fab04a3bf7a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.640517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6280663-e7d1-44a4-bd5b-6fab04a3bf7a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.645601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dad55c-c807-4799-8f18-ec6d2537b03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.646469 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dad55c-c807-4799-8f18-ec6d2537b03a admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.646760 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dad55c-c807-4799-8f18-ec6d2537b03a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.647043 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dad55c-c807-4799-8f18-ec6d2537b03a admin admin] PATH_INFO: `/v3/projects/7aa6e3dfeb894822858eab6b4d1dbce4/users/771bb81a150f426abdfe94738a6706ed/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.661460 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dad55c-c807-4799-8f18-ec6d2537b03a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aa6e3dfeb894822858eab6b4d1dbce4, user_id=771bb81a150f426abdfe94738a6706ed, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.664143 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dad55c-c807-4799-8f18-ec6d2537b03a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.676495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1320/2642] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:47 2022] PUT /identity/v3/projects/7aa6e3dfeb894822858eab6b4d1dbce4/users/771bb81a150f426abdfe94738a6706ed/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:47.687268 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32337b-fcd7-491a-b510-1748e42d82b4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.687568 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32337b-fcd7-491a-b510-1748e42d82b4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.687856 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32337b-fcd7-491a-b510-1748e42d82b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.712678 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:47.715972 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6e32337b-fcd7-491a-b510-1748e42d82b4 None None] MFA Rules not processed for user `771bb81a150f426abdfe94738a6706ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:47.717806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6e32337b-fcd7-491a-b510-1748e42d82b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:47.782185 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1323/2643] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:47.786610 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dad55c-c807-4799-8f18-ec6d2537b03a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.813863 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:47.851002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dad55c-c807-4799-8f18-ec6d2537b03a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.855700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae579d5c-a372-4d40-8ccb-a64a57661ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.856387 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae579d5c-a372-4d40-8ccb-a64a57661ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.856680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae579d5c-a372-4d40-8ccb-a64a57661ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.856966 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae579d5c-a372-4d40-8ccb-a64a57661ee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.857657 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae579d5c-a372-4d40-8ccb-a64a57661ee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.859567 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae579d5c-a372-4d40-8ccb-a64a57661ee3 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.884506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1321/2644] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:47 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:47.888666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32337b-fcd7-491a-b510-1748e42d82b4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.899214 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32337b-fcd7-491a-b510-1748e42d82b4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.955812 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7edb53-bebd-4eee-b4fb-56a40a6173fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.956522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7edb53-bebd-4eee-b4fb-56a40a6173fc admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.956830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7edb53-bebd-4eee-b4fb-56a40a6173fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.957095 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7edb53-bebd-4eee-b4fb-56a40a6173fc admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.957806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7edb53-bebd-4eee-b4fb-56a40a6173fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.959725 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7edb53-bebd-4eee-b4fb-56a40a6173fc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.979503 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1324/2645] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:47 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:47.983794 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae579d5c-a372-4d40-8ccb-a64a57661ee3 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.994325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae579d5c-a372-4d40-8ccb-a64a57661ee3 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.998960 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d030b98-27c1-4fcb-8983-7dfa9a12ba58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.999619 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d030b98-27c1-4fcb-8983-7dfa9a12ba58 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.999919 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d030b98-27c1-4fcb-8983-7dfa9a12ba58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:48.000234 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d030b98-27c1-4fcb-8983-7dfa9a12ba58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:48.000959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d030b98-27c1-4fcb-8983-7dfa9a12ba58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:48.002559 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d030b98-27c1-4fcb-8983-7dfa9a12ba58 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:48.013653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1322/2646] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:48.017671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7edb53-bebd-4eee-b4fb-56a40a6173fc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:48.028180 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7edb53-bebd-4eee-b4fb-56a40a6173fc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:48.032758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0d497-75b4-4b06-953c-0601ef696d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:48.033627 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0d497-75b4-4b06-953c-0601ef696d46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:48.033913 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0d497-75b4-4b06-953c-0601ef696d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:48.034189 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0d497-75b4-4b06-953c-0601ef696d46 admin admin] PATH_INFO: `/v3/projects/855e9216cc4547c1a0856d731e45533c/users/ec00d20a71d44785a0b85f4545e5e78d/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:48.048761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0d497-75b4-4b06-953c-0601ef696d46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=855e9216cc4547c1a0856d731e45533c, user_id=ec00d20a71d44785a0b85f4545e5e78d, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:48.052050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0d497-75b4-4b06-953c-0601ef696d46 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:48.064498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1325/2647] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:48 2022] PUT /identity/v3/projects/855e9216cc4547c1a0856d731e45533c/users/ec00d20a71d44785a0b85f4545e5e78d/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:48.068292 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d030b98-27c1-4fcb-8983-7dfa9a12ba58 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:48.078822 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d030b98-27c1-4fcb-8983-7dfa9a12ba58 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:48.135436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb670d-18c6-4de2-b54a-b9caa435d925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:48.136144 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb670d-18c6-4de2-b54a-b9caa435d925 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:48.136436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb670d-18c6-4de2-b54a-b9caa435d925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:48.136725 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb670d-18c6-4de2-b54a-b9caa435d925 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:48.137457 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb670d-18c6-4de2-b54a-b9caa435d925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:48.139049 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb670d-18c6-4de2-b54a-b9caa435d925 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:48.149145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1323/2648] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:48.153407 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0d497-75b4-4b06-953c-0601ef696d46 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:48.163446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0d497-75b4-4b06-953c-0601ef696d46 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:48.168875 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d3f71-d7b0-4ce8-bd21-d6c4faee518c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:48.169777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d3f71-d7b0-4ce8-bd21-d6c4faee518c admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:48.170201 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d3f71-d7b0-4ce8-bd21-d6c4faee518c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:48.170492 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d3f71-d7b0-4ce8-bd21-d6c4faee518c admin admin] PATH_INFO: `/v3/projects/855e9216cc4547c1a0856d731e45533c/users/ec00d20a71d44785a0b85f4545e5e78d/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:48.172652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d3f71-d7b0-4ce8-bd21-d6c4faee518c admin admin] RBAC: Authorizing `identity:create_grant(project_id=855e9216cc4547c1a0856d731e45533c, user_id=ec00d20a71d44785a0b85f4545e5e78d, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:48.175306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d3f71-d7b0-4ce8-bd21-d6c4faee518c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:48.186918 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1326/2649] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:48 2022] PUT /identity/v3/projects/855e9216cc4547c1a0856d731e45533c/users/ec00d20a71d44785a0b85f4545e5e78d/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:20:48.252587 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14e1d2-e0db-41ef-b30d-23c309d66306 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:48.253132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14e1d2-e0db-41ef-b30d-23c309d66306 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:48.253132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14e1d2-e0db-41ef-b30d-23c309d66306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:48.279602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-0e14e1d2-e0db-41ef-b30d-23c309d66306 None None] MFA Rules not processed for user `ec00d20a71d44785a0b85f4545e5e78d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:48.280875 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0e14e1d2-e0db-41ef-b30d-23c309d66306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:48.347218 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1324/2650] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:48 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:48.358429 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d3f71-d7b0-4ce8-bd21-d6c4faee518c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:48.360210 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-202d3f71-d7b0-4ce8-bd21-d6c4faee518c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:48.417846 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d3f71-d7b0-4ce8-bd21-d6c4faee518c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:48.421457 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53de32b-41a2-4f81-9d90-f1190552b5bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:48.422100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53de32b-41a2-4f81-9d90-f1190552b5bc service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:48.422391 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53de32b-41a2-4f81-9d90-f1190552b5bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:48.422677 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53de32b-41a2-4f81-9d90-f1190552b5bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:48.431192 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:48.483233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53de32b-41a2-4f81-9d90-f1190552b5bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:48.484728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53de32b-41a2-4f81-9d90-f1190552b5bc service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:48.494950 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1327/2651] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:49.236213 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14e1d2-e0db-41ef-b30d-23c309d66306 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:49.237651 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0e14e1d2-e0db-41ef-b30d-23c309d66306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:49.264341 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:49.305600 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14e1d2-e0db-41ef-b30d-23c309d66306 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:49.310524 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d9fe6-ae15-4e4a-8acb-783d6620952c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:49.311528 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d9fe6-ae15-4e4a-8acb-783d6620952c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:49.311822 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d9fe6-ae15-4e4a-8acb-783d6620952c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:49.312139 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d9fe6-ae15-4e4a-8acb-783d6620952c admin admin] PATH_INFO: `/v3/users/afab29f3614f4b439d52dda7f0514bd8` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:49.313339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d9fe6-ae15-4e4a-8acb-783d6620952c admin admin] RBAC: Authorizing `identity:delete_user(user_id=afab29f3614f4b439d52dda7f0514bd8)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:49.315266 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d9fe6-ae15-4e4a-8acb-783d6620952c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:49.363994 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d56d9fe6-ae15-4e4a-8acb-783d6620952c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afab29f3614f4b439d52dda7f0514bd8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:49.371140 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d56d9fe6-ae15-4e4a-8acb-783d6620952c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afab29f3614f4b439d52dda7f0514bd8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:49.378085 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d56d9fe6-ae15-4e4a-8acb-783d6620952c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afab29f3614f4b439d52dda7f0514bd8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:49.378726 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d56d9fe6-ae15-4e4a-8acb-783d6620952c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afab29f3614f4b439d52dda7f0514bd8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:49.391518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d56d9fe6-ae15-4e4a-8acb-783d6620952c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afab29f3614f4b439d52dda7f0514bd8'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:49.397454 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1325/2652] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:49 2022] DELETE /identity/v3/users/afab29f3614f4b439d52dda7f0514bd8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:49.402274 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414bfdb-9679-468f-b786-4f44dc3e997e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:49.402828 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414bfdb-9679-468f-b786-4f44dc3e997e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:49.402940 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4414bfdb-9679-468f-b786-4f44dc3e997e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:49.420656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4414bfdb-9679-468f-b786-4f44dc3e997e None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:49.421990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4414bfdb-9679-468f-b786-4f44dc3e997e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:49.485013 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1328/2653] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:49.502515 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d9fe6-ae15-4e4a-8acb-783d6620952c admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:49.504028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d56d9fe6-ae15-4e4a-8acb-783d6620952c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:49.577941 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d9fe6-ae15-4e4a-8acb-783d6620952c admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:49.582350 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c909aeb-a09e-47da-98c2-3bdba3e2e8e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:49.583194 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c909aeb-a09e-47da-98c2-3bdba3e2e8e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:49.583595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c909aeb-a09e-47da-98c2-3bdba3e2e8e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:49.584035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c909aeb-a09e-47da-98c2-3bdba3e2e8e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:49.594600 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c909aeb-a09e-47da-98c2-3bdba3e2e8e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:49.596188 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c909aeb-a09e-47da-98c2-3bdba3e2e8e2 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:49.607397 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1326/2654] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:49.877103 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414bfdb-9679-468f-b786-4f44dc3e997e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:49.878569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4414bfdb-9679-468f-b786-4f44dc3e997e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:49.890833 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4414bfdb-9679-468f-b786-4f44dc3e997e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:49.898520 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1783fe-f2ad-4d9e-aa57-fbd9194a62e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:49.899495 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1783fe-f2ad-4d9e-aa57-fbd9194a62e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:49.900414 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1783fe-f2ad-4d9e-aa57-fbd9194a62e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:49.900803 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1783fe-f2ad-4d9e-aa57-fbd9194a62e2 admin admin] PATH_INFO: `/v3/projects/9c8e7aaf3c324aff9bcb7dc9f7efcf57` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:49.902112 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1783fe-f2ad-4d9e-aa57-fbd9194a62e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c8e7aaf3c324aff9bcb7dc9f7efcf57)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:49.904182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1783fe-f2ad-4d9e-aa57-fbd9194a62e2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:49.964777 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eb1783fe-f2ad-4d9e-aa57-fbd9194a62e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c8e7aaf3c324aff9bcb7dc9f7efcf57', 'request_id': 'req-eb1783fe-f2ad-4d9e-aa57-fbd9194a62e2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:49.972519 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eb1783fe-f2ad-4d9e-aa57-fbd9194a62e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c8e7aaf3c324aff9bcb7dc9f7efcf57', 'request_id': 'req-eb1783fe-f2ad-4d9e-aa57-fbd9194a62e2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:49.982115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eb1783fe-f2ad-4d9e-aa57-fbd9194a62e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c8e7aaf3c324aff9bcb7dc9f7efcf57', 'request_id': 'req-eb1783fe-f2ad-4d9e-aa57-fbd9194a62e2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:49.994431 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eb1783fe-f2ad-4d9e-aa57-fbd9194a62e2 admin admin] The token cache is being invalidate because project 9c8e7aaf3c324aff9bcb7dc9f7efcf57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:49.994657 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eb1783fe-f2ad-4d9e-aa57-fbd9194a62e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:49.995319 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eb1783fe-f2ad-4d9e-aa57-fbd9194a62e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:49.996884 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1329/2655] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:49 2022] DELETE /identity/v3/projects/9c8e7aaf3c324aff9bcb7dc9f7efcf57 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:50.002430 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ecfd7-67d9-4a10-bc90-6084b18b531b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:50.002724 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ecfd7-67d9-4a10-bc90-6084b18b531b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:50.003002 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ecfd7-67d9-4a10-bc90-6084b18b531b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:50.022042 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-493ecfd7-67d9-4a10-bc90-6084b18b531b None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:50.023380 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-493ecfd7-67d9-4a10-bc90-6084b18b531b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:50.064591 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:50.088915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1327/2656] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:50.096807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793bf0c-6bae-417b-a519-ed2cc55c02a6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:50.097104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793bf0c-6bae-417b-a519-ed2cc55c02a6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:50.097386 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2793bf0c-6bae-417b-a519-ed2cc55c02a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:50.115193 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2793bf0c-6bae-417b-a519-ed2cc55c02a6 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:50.116427 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2793bf0c-6bae-417b-a519-ed2cc55c02a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:50.124310 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1330/2657] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:50.128471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ecfd7-67d9-4a10-bc90-6084b18b531b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:50.138671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ecfd7-67d9-4a10-bc90-6084b18b531b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:50.143380 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd0d43-f7bd-486d-968c-1324b393e3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:50.144065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd0d43-f7bd-486d-968c-1324b393e3b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:50.144363 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd0d43-f7bd-486d-968c-1324b393e3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:50.144650 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd0d43-f7bd-486d-968c-1324b393e3b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:50.145391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd0d43-f7bd-486d-968c-1324b393e3b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:20:50.145702 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd0d43-f7bd-486d-968c-1324b393e3b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:50.147316 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd0d43-f7bd-486d-968c-1324b393e3b2 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:50.159515 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1328/2658] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:20:50.163750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793bf0c-6bae-417b-a519-ed2cc55c02a6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:50.172996 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2793bf0c-6bae-417b-a519-ed2cc55c02a6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:50.177959 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccedbad-d15d-44e2-ae47-a8a1c815b424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:50.178694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccedbad-d15d-44e2-ae47-a8a1c815b424 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:50.178982 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccedbad-d15d-44e2-ae47-a8a1c815b424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:50.179262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccedbad-d15d-44e2-ae47-a8a1c815b424 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:50.179972 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccedbad-d15d-44e2-ae47-a8a1c815b424 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:50.181915 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccedbad-d15d-44e2-ae47-a8a1c815b424 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:50.207289 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1331/2659] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:50 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:50.211248 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd0d43-f7bd-486d-968c-1324b393e3b2 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:50.221268 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd0d43-f7bd-486d-968c-1324b393e3b2 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:50.277857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3841b70-8a43-43ba-be40-8c5bd2d4cd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:50.278504 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3841b70-8a43-43ba-be40-8c5bd2d4cd88 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:50.278739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3841b70-8a43-43ba-be40-8c5bd2d4cd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:50.279021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3841b70-8a43-43ba-be40-8c5bd2d4cd88 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:50.279726 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3841b70-8a43-43ba-be40-8c5bd2d4cd88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:50.281683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3841b70-8a43-43ba-be40-8c5bd2d4cd88 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:50.303097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1329/2660] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:50 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:20:50.307963 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccedbad-d15d-44e2-ae47-a8a1c815b424 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:50.314691 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:50.318369 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccedbad-d15d-44e2-ae47-a8a1c815b424 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:50.324937 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805ff46-328e-4d14-94d6-30d16d7f8940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:50.325841 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805ff46-328e-4d14-94d6-30d16d7f8940 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:50.326573 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805ff46-328e-4d14-94d6-30d16d7f8940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:50.327240 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805ff46-328e-4d14-94d6-30d16d7f8940 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:50.328622 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805ff46-328e-4d14-94d6-30d16d7f8940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:50.331633 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805ff46-328e-4d14-94d6-30d16d7f8940 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:50.344200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1332/2661] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:50.348271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3841b70-8a43-43ba-be40-8c5bd2d4cd88 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:50.359773 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3841b70-8a43-43ba-be40-8c5bd2d4cd88 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:50.364400 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dcf6be-7595-435e-8379-cba5cadda1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:50.365062 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcf6be-7595-435e-8379-cba5cadda1d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:50.365355 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcf6be-7595-435e-8379-cba5cadda1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:50.365639 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcf6be-7595-435e-8379-cba5cadda1d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:50.366373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dcf6be-7595-435e-8379-cba5cadda1d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:50.367987 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dcf6be-7595-435e-8379-cba5cadda1d9 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:50.378359 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1330/2662] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:50.382376 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805ff46-328e-4d14-94d6-30d16d7f8940 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:50.392939 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805ff46-328e-4d14-94d6-30d16d7f8940 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:50.397565 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5f7e2-a1b9-4c7a-a114-09b20601d083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:50.398477 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5f7e2-a1b9-4c7a-a114-09b20601d083 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:50.398763 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5f7e2-a1b9-4c7a-a114-09b20601d083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:50.399043 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5f7e2-a1b9-4c7a-a114-09b20601d083 admin admin] PATH_INFO: `/v3/projects/713970edcede4517b9e9ad6c11aef7c3/users/b75e2f4e68084a0aa6de5687df738b86/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:50.414308 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5f7e2-a1b9-4c7a-a114-09b20601d083 admin admin] RBAC: Authorizing `identity:create_grant(project_id=713970edcede4517b9e9ad6c11aef7c3, user_id=b75e2f4e68084a0aa6de5687df738b86, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:50.417118 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5f7e2-a1b9-4c7a-a114-09b20601d083 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:50.430690 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1333/2663] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:50 2022] PUT /identity/v3/projects/713970edcede4517b9e9ad6c11aef7c3/users/b75e2f4e68084a0aa6de5687df738b86/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:50.441660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3626da5-c8d3-4a52-ac20-6252ebb0c168 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:50.441962 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3626da5-c8d3-4a52-ac20-6252ebb0c168 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:50.442254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3626da5-c8d3-4a52-ac20-6252ebb0c168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:50.469080 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b3626da5-c8d3-4a52-ac20-6252ebb0c168 None None] MFA Rules not processed for user `b75e2f4e68084a0aa6de5687df738b86`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:50.470290 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b3626da5-c8d3-4a52-ac20-6252ebb0c168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:50.533005 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1331/2664] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:50.543395 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5f7e2-a1b9-4c7a-a114-09b20601d083 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:50.545120 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6ba5f7e2-a1b9-4c7a-a114-09b20601d083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:50.603547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5f7e2-a1b9-4c7a-a114-09b20601d083 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:50.607262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab7e76-f443-4538-9477-45216849a76d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:50.607911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab7e76-f443-4538-9477-45216849a76d service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:50.608263 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab7e76-f443-4538-9477-45216849a76d service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:50.608546 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab7e76-f443-4538-9477-45216849a76d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:50.616718 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab7e76-f443-4538-9477-45216849a76d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:50.618157 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab7e76-f443-4538-9477-45216849a76d service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:50.627398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1334/2665] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:55.897232 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd0049-871a-4f78-a3ae-f2148fa557dc None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:55.897756 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd0049-871a-4f78-a3ae-f2148fa557dc None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:55.897866 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd0049-871a-4f78-a3ae-f2148fa557dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:55.915403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c0bd0049-871a-4f78-a3ae-f2148fa557dc None None] MFA Rules not processed for user `35334a0f92254b5e900aad8c51873809`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:55.917267 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c0bd0049-871a-4f78-a3ae-f2148fa557dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:55.949988 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:55.973496 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1332/2666] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:56.285694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab7e76-f443-4538-9477-45216849a76d service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:56.287205 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-83ab7e76-f443-4538-9477-45216849a76d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:56.295422 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd0049-871a-4f78-a3ae-f2148fa557dc None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:56.296909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c0bd0049-871a-4f78-a3ae-f2148fa557dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:56.350936 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab7e76-f443-4538-9477-45216849a76d service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:56.355257 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782a3d6-2053-4968-abeb-0516c1cc6d18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:56.356145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782a3d6-2053-4968-abeb-0516c1cc6d18 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:56.356434 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782a3d6-2053-4968-abeb-0516c1cc6d18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:56.356723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782a3d6-2053-4968-abeb-0516c1cc6d18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:56.358491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6782a3d6-2053-4968-abeb-0516c1cc6d18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:56.367168 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd0049-871a-4f78-a3ae-f2148fa557dc None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:56.369447 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782a3d6-2053-4968-abeb-0516c1cc6d18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:56.370905 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782a3d6-2053-4968-abeb-0516c1cc6d18 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:56.376031 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bea7e8-1c46-435f-8642-e35d1d1fb50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:56.377030 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bea7e8-1c46-435f-8642-e35d1d1fb50a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:56.377226 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bea7e8-1c46-435f-8642-e35d1d1fb50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:56.377508 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bea7e8-1c46-435f-8642-e35d1d1fb50a admin admin] PATH_INFO: `/v3/users/771bb81a150f426abdfe94738a6706ed` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:56.378816 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bea7e8-1c46-435f-8642-e35d1d1fb50a admin admin] RBAC: Authorizing `identity:delete_user(user_id=771bb81a150f426abdfe94738a6706ed)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:56.379684 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1335/2667] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:56.380803 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bea7e8-1c46-435f-8642-e35d1d1fb50a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:56.435844 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-28bea7e8-1c46-435f-8642-e35d1d1fb50a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '771bb81a150f426abdfe94738a6706ed'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:56.446491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-28bea7e8-1c46-435f-8642-e35d1d1fb50a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '771bb81a150f426abdfe94738a6706ed'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:56.455721 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-28bea7e8-1c46-435f-8642-e35d1d1fb50a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '771bb81a150f426abdfe94738a6706ed'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:56.457127 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-28bea7e8-1c46-435f-8642-e35d1d1fb50a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '771bb81a150f426abdfe94738a6706ed'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:56.473078 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-28bea7e8-1c46-435f-8642-e35d1d1fb50a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '771bb81a150f426abdfe94738a6706ed'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:56.481221 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1333/2668] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:56 2022] DELETE /identity/v3/users/771bb81a150f426abdfe94738a6706ed => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:20:56.486151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49929328-e86d-4340-8de2-5577653e84b4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:56.486454 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49929328-e86d-4340-8de2-5577653e84b4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:56.486734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49929328-e86d-4340-8de2-5577653e84b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:56.506120 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-49929328-e86d-4340-8de2-5577653e84b4 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:56.507376 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-49929328-e86d-4340-8de2-5577653e84b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:56.606680 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1336/2669] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:56.617084 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bea7e8-1c46-435f-8642-e35d1d1fb50a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:56.618566 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-28bea7e8-1c46-435f-8642-e35d1d1fb50a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:56.687278 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bea7e8-1c46-435f-8642-e35d1d1fb50a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:56.690811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f94436-d64d-46f5-9082-27437ae5e553 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:56.691455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f94436-d64d-46f5-9082-27437ae5e553 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:56.691740 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f94436-d64d-46f5-9082-27437ae5e553 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:56.692057 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f94436-d64d-46f5-9082-27437ae5e553 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:56.702406 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f94436-d64d-46f5-9082-27437ae5e553 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:56.704025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f94436-d64d-46f5-9082-27437ae5e553 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:56.714022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1334/2670] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:56.906941 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49929328-e86d-4340-8de2-5577653e84b4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:56.908812 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-49929328-e86d-4340-8de2-5577653e84b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:56.929548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:56.988783 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49929328-e86d-4340-8de2-5577653e84b4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:56.993928 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ea2eb-9cd1-4242-a6c2-9f1e6c7b7d5c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd5b2ba9e37041c78a99f1700a4970c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:56.995139 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ea2eb-9cd1-4242-a6c2-9f1e6c7b7d5c service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:56.995430 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ea2eb-9cd1-4242-a6c2-9f1e6c7b7d5c service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:56.995713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ea2eb-9cd1-4242-a6c2-9f1e6c7b7d5c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:56.997710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-940ea2eb-9cd1-4242-a6c2-9f1e6c7b7d5c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:57.010427 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ea2eb-9cd1-4242-a6c2-9f1e6c7b7d5c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:57.012318 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ea2eb-9cd1-4242-a6c2-9f1e6c7b7d5c service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:57.023670 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1337/2671] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:57.035829 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f94436-d64d-46f5-9082-27437ae5e553 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:57.037703 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-33f94436-d64d-46f5-9082-27437ae5e553 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:57.048597 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:57.054198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f94436-d64d-46f5-9082-27437ae5e553 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:57.061219 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2156e8-5652-4950-a42e-07ff0eb1aa87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:57.062091 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2156e8-5652-4950-a42e-07ff0eb1aa87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:57.062392 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2156e8-5652-4950-a42e-07ff0eb1aa87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:57.062828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2156e8-5652-4950-a42e-07ff0eb1aa87 admin admin] PATH_INFO: `/v3/projects/7aa6e3dfeb894822858eab6b4d1dbce4` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:57.065695 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2156e8-5652-4950-a42e-07ff0eb1aa87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aa6e3dfeb894822858eab6b4d1dbce4)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:57.067739 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2156e8-5652-4950-a42e-07ff0eb1aa87 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:57.117113 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6c2156e8-5652-4950-a42e-07ff0eb1aa87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aa6e3dfeb894822858eab6b4d1dbce4', 'request_id': 'req-6c2156e8-5652-4950-a42e-07ff0eb1aa87'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:57.123267 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6c2156e8-5652-4950-a42e-07ff0eb1aa87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aa6e3dfeb894822858eab6b4d1dbce4', 'request_id': 'req-6c2156e8-5652-4950-a42e-07ff0eb1aa87'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:57.132989 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6c2156e8-5652-4950-a42e-07ff0eb1aa87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aa6e3dfeb894822858eab6b4d1dbce4', 'request_id': 'req-6c2156e8-5652-4950-a42e-07ff0eb1aa87'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:57.146940 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6c2156e8-5652-4950-a42e-07ff0eb1aa87 admin admin] The token cache is being invalidate because project 7aa6e3dfeb894822858eab6b4d1dbce4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:57.147239 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6c2156e8-5652-4950-a42e-07ff0eb1aa87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:57.147919 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-6c2156e8-5652-4950-a42e-07ff0eb1aa87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:57.149522 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1335/2672] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:57 2022] DELETE /identity/v3/projects/7aa6e3dfeb894822858eab6b4d1dbce4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:20:57.153840 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ea2eb-9cd1-4242-a6c2-9f1e6c7b7d5c service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:57.164853 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ea2eb-9cd1-4242-a6c2-9f1e6c7b7d5c service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:57.224847 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a812c9-ce18-4b16-8bb7-78135b08bd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:57.225888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a812c9-ce18-4b16-8bb7-78135b08bd90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:57.226207 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a812c9-ce18-4b16-8bb7-78135b08bd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:57.226490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a812c9-ce18-4b16-8bb7-78135b08bd90 admin admin] PATH_INFO: `/v3/users/ec00d20a71d44785a0b85f4545e5e78d` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:57.227720 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a812c9-ce18-4b16-8bb7-78135b08bd90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec00d20a71d44785a0b85f4545e5e78d)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:57.230362 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a812c9-ce18-4b16-8bb7-78135b08bd90 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:57.281292 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-35a812c9-ce18-4b16-8bb7-78135b08bd90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec00d20a71d44785a0b85f4545e5e78d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:57.289116 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-35a812c9-ce18-4b16-8bb7-78135b08bd90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec00d20a71d44785a0b85f4545e5e78d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:57.297122 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-35a812c9-ce18-4b16-8bb7-78135b08bd90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec00d20a71d44785a0b85f4545e5e78d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:57.298031 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-35a812c9-ce18-4b16-8bb7-78135b08bd90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec00d20a71d44785a0b85f4545e5e78d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:57.310982 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-35a812c9-ce18-4b16-8bb7-78135b08bd90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec00d20a71d44785a0b85f4545e5e78d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:57.317460 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1338/2673] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:57 2022] DELETE /identity/v3/users/ec00d20a71d44785a0b85f4545e5e78d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:20:57.849675 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2156e8-5652-4950-a42e-07ff0eb1aa87 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:57.851200 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6c2156e8-5652-4950-a42e-07ff0eb1aa87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:57.871910 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2156e8-5652-4950-a42e-07ff0eb1aa87 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:57.959422 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b58259-e65a-474a-a6c0-a3f4f9452032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:57.960422 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b58259-e65a-474a-a6c0-a3f4f9452032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:57.960713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b58259-e65a-474a-a6c0-a3f4f9452032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:57.961020 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b58259-e65a-474a-a6c0-a3f4f9452032 admin admin] PATH_INFO: `/v3/projects/855e9216cc4547c1a0856d731e45533c` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:57.962236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b58259-e65a-474a-a6c0-a3f4f9452032 admin admin] RBAC: Authorizing `identity:delete_project(project_id=855e9216cc4547c1a0856d731e45533c)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:57.964736 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b58259-e65a-474a-a6c0-a3f4f9452032 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:58.031473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-b4b58259-e65a-474a-a6c0-a3f4f9452032 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '855e9216cc4547c1a0856d731e45533c', 'request_id': 'req-b4b58259-e65a-474a-a6c0-a3f4f9452032'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:58.037913 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-b4b58259-e65a-474a-a6c0-a3f4f9452032 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '855e9216cc4547c1a0856d731e45533c', 'request_id': 'req-b4b58259-e65a-474a-a6c0-a3f4f9452032'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:58.047011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-b4b58259-e65a-474a-a6c0-a3f4f9452032 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '855e9216cc4547c1a0856d731e45533c', 'request_id': 'req-b4b58259-e65a-474a-a6c0-a3f4f9452032'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:58.064577 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-b4b58259-e65a-474a-a6c0-a3f4f9452032 admin admin] The token cache is being invalidate because project 855e9216cc4547c1a0856d731e45533c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:58.064878 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-b4b58259-e65a-474a-a6c0-a3f4f9452032 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:58.065608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-b4b58259-e65a-474a-a6c0-a3f4f9452032 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:58.067193 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1336/2674] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:57 2022] DELETE /identity/v3/projects/855e9216cc4547c1a0856d731e45533c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:58.072340 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38df872-c6e5-4409-ab51-ba6abb419c31 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:58.072652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38df872-c6e5-4409-ab51-ba6abb419c31 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:58.072936 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38df872-c6e5-4409-ab51-ba6abb419c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:58.104332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e38df872-c6e5-4409-ab51-ba6abb419c31 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:58.105603 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e38df872-c6e5-4409-ab51-ba6abb419c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:58.126657 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:58.174217 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1339/2675] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:20:58.182825 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d563f-3d99-4958-b72f-4d473ff7a84d None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:58.183122 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d563f-3d99-4958-b72f-4d473ff7a84d None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:58.183406 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d563f-3d99-4958-b72f-4d473ff7a84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:58.202313 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ad7d563f-3d99-4958-b72f-4d473ff7a84d None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:58.203545 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ad7d563f-3d99-4958-b72f-4d473ff7a84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:58.211636 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1337/2676] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:58.215907 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38df872-c6e5-4409-ab51-ba6abb419c31 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:58.226001 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38df872-c6e5-4409-ab51-ba6abb419c31 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:58.231506 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd04f16-01cc-44e6-9443-2e7cce5fb37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:58.232332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd04f16-01cc-44e6-9443-2e7cce5fb37c admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:58.232498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd04f16-01cc-44e6-9443-2e7cce5fb37c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:58.232840 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd04f16-01cc-44e6-9443-2e7cce5fb37c admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:58.233619 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd04f16-01cc-44e6-9443-2e7cce5fb37c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:20:58.234262 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd04f16-01cc-44e6-9443-2e7cce5fb37c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:58.235634 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd04f16-01cc-44e6-9443-2e7cce5fb37c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:58.249845 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1340/2677] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:20:58.254068 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d563f-3d99-4958-b72f-4d473ff7a84d None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:58.265464 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d563f-3d99-4958-b72f-4d473ff7a84d None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:58.271338 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d049f-2cfc-42c8-b988-760e705e84ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:58.272504 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d049f-2cfc-42c8-b988-760e705e84ad admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:58.272504 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d049f-2cfc-42c8-b988-760e705e84ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:58.272687 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d049f-2cfc-42c8-b988-760e705e84ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:58.273329 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d049f-2cfc-42c8-b988-760e705e84ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:58.275232 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d049f-2cfc-42c8-b988-760e705e84ad admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:58.284214 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:58.306188 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1338/2678] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:58 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:20:58.310673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd04f16-01cc-44e6-9443-2e7cce5fb37c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:58.321196 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd04f16-01cc-44e6-9443-2e7cce5fb37c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:58.379806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4f12b-d84d-45e2-8d00-b0dfbc19353f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:58.380516 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4f12b-d84d-45e2-8d00-b0dfbc19353f admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:58.380807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4f12b-d84d-45e2-8d00-b0dfbc19353f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:58.381092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4f12b-d84d-45e2-8d00-b0dfbc19353f admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:58.381782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4f12b-d84d-45e2-8d00-b0dfbc19353f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:58.383734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4f12b-d84d-45e2-8d00-b0dfbc19353f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:58.407736 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1341/2679] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:58 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:58.408922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d049f-2cfc-42c8-b988-760e705e84ad admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:58.420043 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d049f-2cfc-42c8-b988-760e705e84ad admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:58.425009 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580e151-fd5e-42ec-8c26-1b75616f7158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:58.425751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580e151-fd5e-42ec-8c26-1b75616f7158 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:58.426120 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580e151-fd5e-42ec-8c26-1b75616f7158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:58.426471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580e151-fd5e-42ec-8c26-1b75616f7158 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:58.427281 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580e151-fd5e-42ec-8c26-1b75616f7158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:58.429258 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580e151-fd5e-42ec-8c26-1b75616f7158 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:58.440001 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1339/2680] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:58.444040 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4f12b-d84d-45e2-8d00-b0dfbc19353f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:58.454488 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4f12b-d84d-45e2-8d00-b0dfbc19353f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:58.459377 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9db724-b37f-4901-93bf-81da8b2bf00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:58.460114 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9db724-b37f-4901-93bf-81da8b2bf00b admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:58.460403 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9db724-b37f-4901-93bf-81da8b2bf00b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:58.460680 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9db724-b37f-4901-93bf-81da8b2bf00b admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:58.461403 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9db724-b37f-4901-93bf-81da8b2bf00b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:58.463569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9db724-b37f-4901-93bf-81da8b2bf00b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:58.473540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1342/2681] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:58.477859 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580e151-fd5e-42ec-8c26-1b75616f7158 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:58.489312 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580e151-fd5e-42ec-8c26-1b75616f7158 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:58.494084 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d607ed1-931e-4d79-977c-0150857c1578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:58.495017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d607ed1-931e-4d79-977c-0150857c1578 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:58.495304 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d607ed1-931e-4d79-977c-0150857c1578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:58.495587 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d607ed1-931e-4d79-977c-0150857c1578 admin admin] PATH_INFO: `/v3/projects/c6a331b288da434c8c078433a3ab0ac8/users/30820a065178430996c150e3fd62579d/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:58.511286 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d607ed1-931e-4d79-977c-0150857c1578 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a331b288da434c8c078433a3ab0ac8, user_id=30820a065178430996c150e3fd62579d, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:58.514026 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d607ed1-931e-4d79-977c-0150857c1578 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:58.527933 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1340/2682] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:58 2022] PUT /identity/v3/projects/c6a331b288da434c8c078433a3ab0ac8/users/30820a065178430996c150e3fd62579d/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:58.538268 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28466405-6daa-4bd9-a8a2-958b734f5298 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:58.538574 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28466405-6daa-4bd9-a8a2-958b734f5298 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:58.538873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28466405-6daa-4bd9-a8a2-958b734f5298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:58.567911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-28466405-6daa-4bd9-a8a2-958b734f5298 None None] MFA Rules not processed for user `30820a065178430996c150e3fd62579d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:58.569805 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-28466405-6daa-4bd9-a8a2-958b734f5298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:58.636003 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1343/2683] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:58.640355 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d607ed1-931e-4d79-977c-0150857c1578 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:58.704291 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d607ed1-931e-4d79-977c-0150857c1578 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:58.709175 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96361e79-131b-4059-b9f7-778462396195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:58.709873 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96361e79-131b-4059-b9f7-778462396195 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:58.710113 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96361e79-131b-4059-b9f7-778462396195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:58.710388 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96361e79-131b-4059-b9f7-778462396195 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:58.711088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96361e79-131b-4059-b9f7-778462396195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:58.713696 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96361e79-131b-4059-b9f7-778462396195 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:58.740467 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1341/2684] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:58 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:20:58.744828 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28466405-6daa-4bd9-a8a2-958b734f5298 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:58.758211 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28466405-6daa-4bd9-a8a2-958b734f5298 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:58.778701 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:58.835287 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807ee83-6380-49e9-b755-dd418f54d6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:58.836194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807ee83-6380-49e9-b755-dd418f54d6f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:58.836507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807ee83-6380-49e9-b755-dd418f54d6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:58.836788 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807ee83-6380-49e9-b755-dd418f54d6f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:58.837487 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807ee83-6380-49e9-b755-dd418f54d6f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:58.840325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807ee83-6380-49e9-b755-dd418f54d6f4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:58.863607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1344/2685] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:58 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:58.867676 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96361e79-131b-4059-b9f7-778462396195 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:58.882941 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96361e79-131b-4059-b9f7-778462396195 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:58.889000 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47de02-ff58-4c25-98c7-d3545bfdb8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:58.889745 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47de02-ff58-4c25-98c7-d3545bfdb8d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:58.890046 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47de02-ff58-4c25-98c7-d3545bfdb8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:58.890329 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47de02-ff58-4c25-98c7-d3545bfdb8d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:58.891229 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b47de02-ff58-4c25-98c7-d3545bfdb8d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:58.892899 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b47de02-ff58-4c25-98c7-d3545bfdb8d6 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:58.910698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1342/2686] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:58.911970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807ee83-6380-49e9-b755-dd418f54d6f4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:58.924063 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807ee83-6380-49e9-b755-dd418f54d6f4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:58.929386 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d20ee-8ec3-44f7-a12a-c4330f82b021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:58.930078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d20ee-8ec3-44f7-a12a-c4330f82b021 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:58.930370 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d20ee-8ec3-44f7-a12a-c4330f82b021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:58.930652 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d20ee-8ec3-44f7-a12a-c4330f82b021 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:58.931441 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412d20ee-8ec3-44f7-a12a-c4330f82b021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:58.933112 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412d20ee-8ec3-44f7-a12a-c4330f82b021 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:58.955671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1345/2687] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:58.959528 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47de02-ff58-4c25-98c7-d3545bfdb8d6 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:58.970311 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47de02-ff58-4c25-98c7-d3545bfdb8d6 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:58.976327 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625c0f9-140d-4830-9ad9-53176ebb8a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:58.977228 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625c0f9-140d-4830-9ad9-53176ebb8a9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:58.977527 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625c0f9-140d-4830-9ad9-53176ebb8a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:58.977811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625c0f9-140d-4830-9ad9-53176ebb8a9e admin admin] PATH_INFO: `/v3/projects/2f8a0f6175764741bf7789b453d19e04/users/acafa08d2a14480184a89f2bd3dd7b94/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:58.993241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625c0f9-140d-4830-9ad9-53176ebb8a9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f8a0f6175764741bf7789b453d19e04, user_id=acafa08d2a14480184a89f2bd3dd7b94, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:58.996346 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625c0f9-140d-4830-9ad9-53176ebb8a9e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:59.011287 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1343/2688] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:58 2022] PUT /identity/v3/projects/2f8a0f6175764741bf7789b453d19e04/users/acafa08d2a14480184a89f2bd3dd7b94/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:59.023504 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7fa97-5cac-4f9f-ab97-ae9a9dbeaba3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:59.023808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7fa97-5cac-4f9f-ab97-ae9a9dbeaba3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:59.024670 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7fa97-5cac-4f9f-ab97-ae9a9dbeaba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:59.067271 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-dff7fa97-5cac-4f9f-ab97-ae9a9dbeaba3 None None] MFA Rules not processed for user `acafa08d2a14480184a89f2bd3dd7b94`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:59.069220 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dff7fa97-5cac-4f9f-ab97-ae9a9dbeaba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:59.177673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1346/2689] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:20:59.183022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625c0f9-140d-4830-9ad9-53176ebb8a9e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:59.268595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625c0f9-140d-4830-9ad9-53176ebb8a9e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:59.275108 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2618cfc-2cb2-404b-94d7-f7e769053116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:59.275886 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2618cfc-2cb2-404b-94d7-f7e769053116 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:59.276212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2618cfc-2cb2-404b-94d7-f7e769053116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:59.277881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2618cfc-2cb2-404b-94d7-f7e769053116 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:59.277881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2618cfc-2cb2-404b-94d7-f7e769053116 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:59.280178 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2618cfc-2cb2-404b-94d7-f7e769053116 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:59.291484 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:59.321444 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1344/2690] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:59 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:20:59.327194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7fa97-5cac-4f9f-ab97-ae9a9dbeaba3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:59.341497 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7fa97-5cac-4f9f-ab97-ae9a9dbeaba3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:59.445581 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f4d59-c8bb-483d-8845-e689ed0b8b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:59.446280 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f4d59-c8bb-483d-8845-e689ed0b8b18 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:59.446568 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f4d59-c8bb-483d-8845-e689ed0b8b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:59.446846 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f4d59-c8bb-483d-8845-e689ed0b8b18 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:59.448162 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f4d59-c8bb-483d-8845-e689ed0b8b18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:59.450295 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f4d59-c8bb-483d-8845-e689ed0b8b18 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:59.478586 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1347/2691] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:59 2022] POST /identity/v3/users => generated 298 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:59.482670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2618cfc-2cb2-404b-94d7-f7e769053116 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:59.498038 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2618cfc-2cb2-404b-94d7-f7e769053116 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:59.511898 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0effe-c15b-4abc-b139-e672d397307c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:59.512742 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0effe-c15b-4abc-b139-e672d397307c admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:59.513448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0effe-c15b-4abc-b139-e672d397307c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:59.513448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0effe-c15b-4abc-b139-e672d397307c admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:59.514247 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e0effe-c15b-4abc-b139-e672d397307c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:59.516216 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e0effe-c15b-4abc-b139-e672d397307c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:59.543436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1345/2692] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:59.544887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f4d59-c8bb-483d-8845-e689ed0b8b18 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:59.556259 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f4d59-c8bb-483d-8845-e689ed0b8b18 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:59.563656 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b94fe6-0e10-48d4-8afa-ac161ad7cc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:59.564840 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b94fe6-0e10-48d4-8afa-ac161ad7cc9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:59.566386 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b94fe6-0e10-48d4-8afa-ac161ad7cc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:59.566737 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b94fe6-0e10-48d4-8afa-ac161ad7cc9a admin admin] PATH_INFO: `/v3/projects/3a67f5d9b4b042ed929fa5c3d2e07276/users/6b7148c8084f47dcbe5d81cdac275462/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:59.585087 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b94fe6-0e10-48d4-8afa-ac161ad7cc9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a67f5d9b4b042ed929fa5c3d2e07276, user_id=6b7148c8084f47dcbe5d81cdac275462, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:59.590562 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b94fe6-0e10-48d4-8afa-ac161ad7cc9a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:59.594272 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:59.613827 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1348/2693] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:59 2022] PUT /identity/v3/projects/3a67f5d9b4b042ed929fa5c3d2e07276/users/6b7148c8084f47dcbe5d81cdac275462/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:59.615102 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0effe-c15b-4abc-b139-e672d397307c admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:59.634809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0effe-c15b-4abc-b139-e672d397307c admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:59.709154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc85f3f4-3340-470f-88eb-4bb762d25820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:59.709851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85f3f4-3340-470f-88eb-4bb762d25820 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:59.710139 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85f3f4-3340-470f-88eb-4bb762d25820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:59.710425 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc85f3f4-3340-470f-88eb-4bb762d25820 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:59.711166 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc85f3f4-3340-470f-88eb-4bb762d25820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:59.712934 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc85f3f4-3340-470f-88eb-4bb762d25820 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:59.737306 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1346/2694] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:59.742077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b94fe6-0e10-48d4-8afa-ac161ad7cc9a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:59.783269 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b94fe6-0e10-48d4-8afa-ac161ad7cc9a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:59.789006 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437335f2-17c5-4e8b-85f4-e929be4e81b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:59.789914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437335f2-17c5-4e8b-85f4-e929be4e81b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:59.790206 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437335f2-17c5-4e8b-85f4-e929be4e81b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:59.790487 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437335f2-17c5-4e8b-85f4-e929be4e81b9 admin admin] PATH_INFO: `/v3/projects/3a67f5d9b4b042ed929fa5c3d2e07276/users/6b7148c8084f47dcbe5d81cdac275462/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:59.793103 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437335f2-17c5-4e8b-85f4-e929be4e81b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a67f5d9b4b042ed929fa5c3d2e07276, user_id=6b7148c8084f47dcbe5d81cdac275462, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:59.795790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437335f2-17c5-4e8b-85f4-e929be4e81b9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:59.811759 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1349/2695] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:59 2022] PUT /identity/v3/projects/3a67f5d9b4b042ed929fa5c3d2e07276/users/6b7148c8084f47dcbe5d81cdac275462/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:59.822979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf104060-bea2-4d93-807e-4cea9a2e2b15 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:59.823534 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf104060-bea2-4d93-807e-4cea9a2e2b15 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:59.824021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf104060-bea2-4d93-807e-4cea9a2e2b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:59.865088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-bf104060-bea2-4d93-807e-4cea9a2e2b15 None None] MFA Rules not processed for user `6b7148c8084f47dcbe5d81cdac275462`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:59.867305 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bf104060-bea2-4d93-807e-4cea9a2e2b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:59.976115 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1347/2696] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:20:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:59.985252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437335f2-17c5-4e8b-85f4-e929be4e81b9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:59.987094 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-437335f2-17c5-4e8b-85f4-e929be4e81b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:00.058488 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437335f2-17c5-4e8b-85f4-e929be4e81b9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:00.062260 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631fca31-36e2-4c49-9a81-41b9f31e9ed9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:00.062920 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631fca31-36e2-4c49-9a81-41b9f31e9ed9 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:00.063211 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631fca31-36e2-4c49-9a81-41b9f31e9ed9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:00.063750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631fca31-36e2-4c49-9a81-41b9f31e9ed9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:00.132076 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631fca31-36e2-4c49-9a81-41b9f31e9ed9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:00.133544 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631fca31-36e2-4c49-9a81-41b9f31e9ed9 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:00.143410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1350/2697] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:21:02.723491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf104060-bea2-4d93-807e-4cea9a2e2b15 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:02.734845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf104060-bea2-4d93-807e-4cea9a2e2b15 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:02.738407 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca4f2a-7d3c-4125-b70f-7c3c1d9c6c79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:02.739057 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca4f2a-7d3c-4125-b70f-7c3c1d9c6c79 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:02.739346 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca4f2a-7d3c-4125-b70f-7c3c1d9c6c79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:02.739630 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca4f2a-7d3c-4125-b70f-7c3c1d9c6c79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:02.799535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca4f2a-7d3c-4125-b70f-7c3c1d9c6c79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:02.800936 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca4f2a-7d3c-4125-b70f-7c3c1d9c6c79 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:02.804814 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:02.812527 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1348/2698] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:21:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:21:03.337481 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631fca31-36e2-4c49-9a81-41b9f31e9ed9 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:03.346485 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631fca31-36e2-4c49-9a81-41b9f31e9ed9 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:03.350166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80562d-805a-4bc1-a198-35d73b4b2f33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:03.350801 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80562d-805a-4bc1-a198-35d73b4b2f33 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:03.351090 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80562d-805a-4bc1-a198-35d73b4b2f33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:03.351369 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80562d-805a-4bc1-a198-35d73b4b2f33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:03.360827 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80562d-805a-4bc1-a198-35d73b4b2f33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:03.362256 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80562d-805a-4bc1-a198-35d73b4b2f33 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:03.371584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1351/2699] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:21:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:21:04.893705 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca4f2a-7d3c-4125-b70f-7c3c1d9c6c79 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:04.895118 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d2ca4f2a-7d3c-4125-b70f-7c3c1d9c6c79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:04.959915 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca4f2a-7d3c-4125-b70f-7c3c1d9c6c79 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:04.964908 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef242392-f856-4955-b56d-01e64f155a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:04.965767 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef242392-f856-4955-b56d-01e64f155a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:04.966051 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef242392-f856-4955-b56d-01e64f155a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:04.966326 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef242392-f856-4955-b56d-01e64f155a7e admin admin] PATH_INFO: `/v3/users/b75e2f4e68084a0aa6de5687df738b86` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:04.967501 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef242392-f856-4955-b56d-01e64f155a7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b75e2f4e68084a0aa6de5687df738b86)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:04.969447 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef242392-f856-4955-b56d-01e64f155a7e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:05.014080 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ef242392-f856-4955-b56d-01e64f155a7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b75e2f4e68084a0aa6de5687df738b86'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:05.020490 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ef242392-f856-4955-b56d-01e64f155a7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b75e2f4e68084a0aa6de5687df738b86'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:05.027737 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ef242392-f856-4955-b56d-01e64f155a7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b75e2f4e68084a0aa6de5687df738b86'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:05.028378 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ef242392-f856-4955-b56d-01e64f155a7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b75e2f4e68084a0aa6de5687df738b86'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:05.040040 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ef242392-f856-4955-b56d-01e64f155a7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b75e2f4e68084a0aa6de5687df738b86'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:05.046333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1349/2700] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:21:04 2022] DELETE /identity/v3/users/b75e2f4e68084a0aa6de5687df738b86 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:05.050937 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96874a02-f9b3-4031-ad1a-ca1b8445b251 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:05.051435 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96874a02-f9b3-4031-ad1a-ca1b8445b251 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:05.051540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96874a02-f9b3-4031-ad1a-ca1b8445b251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:05.069161 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-96874a02-f9b3-4031-ad1a-ca1b8445b251 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:05.070409 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-96874a02-f9b3-4031-ad1a-ca1b8445b251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:05.136106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1352/2701] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:05.145927 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef242392-f856-4955-b56d-01e64f155a7e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:05.147321 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ef242392-f856-4955-b56d-01e64f155a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:05.211937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef242392-f856-4955-b56d-01e64f155a7e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:05.215611 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638689ab-408f-45fa-b4e4-b4027660faab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:05.216291 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638689ab-408f-45fa-b4e4-b4027660faab service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:05.216580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638689ab-408f-45fa-b4e4-b4027660faab service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:05.216865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638689ab-408f-45fa-b4e4-b4027660faab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:05.225750 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638689ab-408f-45fa-b4e4-b4027660faab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:05.227175 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638689ab-408f-45fa-b4e4-b4027660faab service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:05.237123 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1350/2702] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:21:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:21:05.605436 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96874a02-f9b3-4031-ad1a-ca1b8445b251 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:05.606992 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-96874a02-f9b3-4031-ad1a-ca1b8445b251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:05.620324 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96874a02-f9b3-4031-ad1a-ca1b8445b251 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:05.625143 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d641a-5abe-4b7f-aab1-861dda49bc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:05.626186 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d641a-5abe-4b7f-aab1-861dda49bc7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:05.626468 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d641a-5abe-4b7f-aab1-861dda49bc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:05.626818 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d641a-5abe-4b7f-aab1-861dda49bc7c admin admin] PATH_INFO: `/v3/projects/713970edcede4517b9e9ad6c11aef7c3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:05.628750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d641a-5abe-4b7f-aab1-861dda49bc7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=713970edcede4517b9e9ad6c11aef7c3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:05.630866 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d641a-5abe-4b7f-aab1-861dda49bc7c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:05.650038 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:05.681814 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-df7d641a-5abe-4b7f-aab1-861dda49bc7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '713970edcede4517b9e9ad6c11aef7c3', 'request_id': 'req-df7d641a-5abe-4b7f-aab1-861dda49bc7c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:05.687892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-df7d641a-5abe-4b7f-aab1-861dda49bc7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '713970edcede4517b9e9ad6c11aef7c3', 'request_id': 'req-df7d641a-5abe-4b7f-aab1-861dda49bc7c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:05.694891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-df7d641a-5abe-4b7f-aab1-861dda49bc7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '713970edcede4517b9e9ad6c11aef7c3', 'request_id': 'req-df7d641a-5abe-4b7f-aab1-861dda49bc7c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:05.705636 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-df7d641a-5abe-4b7f-aab1-861dda49bc7c admin admin] The token cache is being invalidate because project 713970edcede4517b9e9ad6c11aef7c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:21:05.705927 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-df7d641a-5abe-4b7f-aab1-861dda49bc7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:05.706567 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-df7d641a-5abe-4b7f-aab1-861dda49bc7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:05.708139 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1353/2703] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:21:05 2022] DELETE /identity/v3/projects/713970edcede4517b9e9ad6c11aef7c3 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:07.207119 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638689ab-408f-45fa-b4e4-b4027660faab service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:07.208582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-638689ab-408f-45fa-b4e4-b4027660faab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:07.221535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638689ab-408f-45fa-b4e4-b4027660faab service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:07.238483 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:07.285375 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe223385-ce4a-47be-8ecd-e9afc8ec8812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:07.286249 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe223385-ce4a-47be-8ecd-e9afc8ec8812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:07.286531 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe223385-ce4a-47be-8ecd-e9afc8ec8812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:07.286806 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe223385-ce4a-47be-8ecd-e9afc8ec8812 admin admin] PATH_INFO: `/v3/users/30820a065178430996c150e3fd62579d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:07.288019 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe223385-ce4a-47be-8ecd-e9afc8ec8812 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30820a065178430996c150e3fd62579d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:07.289922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe223385-ce4a-47be-8ecd-e9afc8ec8812 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:07.335196 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-fe223385-ce4a-47be-8ecd-e9afc8ec8812 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30820a065178430996c150e3fd62579d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:07.342291 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-fe223385-ce4a-47be-8ecd-e9afc8ec8812 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30820a065178430996c150e3fd62579d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:07.349191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-fe223385-ce4a-47be-8ecd-e9afc8ec8812 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30820a065178430996c150e3fd62579d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:07.349823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-fe223385-ce4a-47be-8ecd-e9afc8ec8812 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30820a065178430996c150e3fd62579d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:07.363163 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-fe223385-ce4a-47be-8ecd-e9afc8ec8812 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30820a065178430996c150e3fd62579d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:07.369276 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1351/2704] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:21:07 2022] DELETE /identity/v3/users/30820a065178430996c150e3fd62579d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:07.374264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27955569-8965-4885-b8c5-caa1118d6810 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:07.374735 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27955569-8965-4885-b8c5-caa1118d6810 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:07.374836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27955569-8965-4885-b8c5-caa1118d6810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:07.392188 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-27955569-8965-4885-b8c5-caa1118d6810 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:07.393400 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-27955569-8965-4885-b8c5-caa1118d6810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:07.451078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1354/2705] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:21:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:07.460557 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe223385-ce4a-47be-8ecd-e9afc8ec8812 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:07.461958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-fe223385-ce4a-47be-8ecd-e9afc8ec8812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:07.518969 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe223385-ce4a-47be-8ecd-e9afc8ec8812 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:07.522616 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f002524-c73f-41d3-bc61-6bd6da7bb16a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:07.523294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f002524-c73f-41d3-bc61-6bd6da7bb16a service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:07.523587 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f002524-c73f-41d3-bc61-6bd6da7bb16a service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:07.523872 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f002524-c73f-41d3-bc61-6bd6da7bb16a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:07.532948 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f002524-c73f-41d3-bc61-6bd6da7bb16a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:07.534379 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f002524-c73f-41d3-bc61-6bd6da7bb16a service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:07.545135 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1352/2706] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:21:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:21:07.810498 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27955569-8965-4885-b8c5-caa1118d6810 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:07.812038 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-27955569-8965-4885-b8c5-caa1118d6810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:07.825020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27955569-8965-4885-b8c5-caa1118d6810 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:07.830286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24b046-8925-438b-955c-13c3ca281d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:07.831312 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24b046-8925-438b-955c-13c3ca281d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:07.831667 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24b046-8925-438b-955c-13c3ca281d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:07.832052 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24b046-8925-438b-955c-13c3ca281d33 admin admin] PATH_INFO: `/v3/projects/c6a331b288da434c8c078433a3ab0ac8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:07.833458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24b046-8925-438b-955c-13c3ca281d33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6a331b288da434c8c078433a3ab0ac8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:07.835447 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24b046-8925-438b-955c-13c3ca281d33 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:07.891681 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9d24b046-8925-438b-955c-13c3ca281d33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6a331b288da434c8c078433a3ab0ac8', 'request_id': 'req-9d24b046-8925-438b-955c-13c3ca281d33'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:07.898095 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9d24b046-8925-438b-955c-13c3ca281d33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6a331b288da434c8c078433a3ab0ac8', 'request_id': 'req-9d24b046-8925-438b-955c-13c3ca281d33'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:07.907790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9d24b046-8925-438b-955c-13c3ca281d33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6a331b288da434c8c078433a3ab0ac8', 'request_id': 'req-9d24b046-8925-438b-955c-13c3ca281d33'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:07.920056 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9d24b046-8925-438b-955c-13c3ca281d33 admin admin] The token cache is being invalidate because project c6a331b288da434c8c078433a3ab0ac8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:21:07.920354 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9d24b046-8925-438b-955c-13c3ca281d33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:07.921000 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9d24b046-8925-438b-955c-13c3ca281d33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:07.922556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1355/2707] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:21:07 2022] DELETE /identity/v3/projects/c6a331b288da434c8c078433a3ab0ac8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:21:07.926627 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f002524-c73f-41d3-bc61-6bd6da7bb16a service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:07.937334 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f002524-c73f-41d3-bc61-6bd6da7bb16a service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:07.978094 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:07.996065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f13821-aa44-4b0f-a7ef-f6a4b19f5d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:07.996889 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f13821-aa44-4b0f-a7ef-f6a4b19f5d1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:07.997182 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f13821-aa44-4b0f-a7ef-f6a4b19f5d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:07.997469 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f13821-aa44-4b0f-a7ef-f6a4b19f5d1e admin admin] PATH_INFO: `/v3/users/acafa08d2a14480184a89f2bd3dd7b94` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:07.998685 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f13821-aa44-4b0f-a7ef-f6a4b19f5d1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=acafa08d2a14480184a89f2bd3dd7b94)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:08.000684 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f13821-aa44-4b0f-a7ef-f6a4b19f5d1e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:08.047447 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-12f13821-aa44-4b0f-a7ef-f6a4b19f5d1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acafa08d2a14480184a89f2bd3dd7b94'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:08.054758 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-12f13821-aa44-4b0f-a7ef-f6a4b19f5d1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acafa08d2a14480184a89f2bd3dd7b94'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:08.061658 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-12f13821-aa44-4b0f-a7ef-f6a4b19f5d1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acafa08d2a14480184a89f2bd3dd7b94'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:08.062320 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-12f13821-aa44-4b0f-a7ef-f6a4b19f5d1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acafa08d2a14480184a89f2bd3dd7b94'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:08.073984 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-12f13821-aa44-4b0f-a7ef-f6a4b19f5d1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acafa08d2a14480184a89f2bd3dd7b94'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:08.079795 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1353/2708] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:21:07 2022] DELETE /identity/v3/users/acafa08d2a14480184a89f2bd3dd7b94 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:08.351548 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24b046-8925-438b-955c-13c3ca281d33 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:08.353120 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9d24b046-8925-438b-955c-13c3ca281d33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:08.366568 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24b046-8925-438b-955c-13c3ca281d33 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:08.394734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:08.426428 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35a84d-73ff-4bf6-a95e-ab5427f092da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:08.427291 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35a84d-73ff-4bf6-a95e-ab5427f092da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:08.427579 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35a84d-73ff-4bf6-a95e-ab5427f092da admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:08.427862 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35a84d-73ff-4bf6-a95e-ab5427f092da admin admin] PATH_INFO: `/v3/projects/2f8a0f6175764741bf7789b453d19e04` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:08.429346 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35a84d-73ff-4bf6-a95e-ab5427f092da admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f8a0f6175764741bf7789b453d19e04)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:08.431246 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35a84d-73ff-4bf6-a95e-ab5427f092da admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:08.476086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1e35a84d-73ff-4bf6-a95e-ab5427f092da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f8a0f6175764741bf7789b453d19e04', 'request_id': 'req-1e35a84d-73ff-4bf6-a95e-ab5427f092da'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:08.482024 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1e35a84d-73ff-4bf6-a95e-ab5427f092da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f8a0f6175764741bf7789b453d19e04', 'request_id': 'req-1e35a84d-73ff-4bf6-a95e-ab5427f092da'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:08.489567 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1e35a84d-73ff-4bf6-a95e-ab5427f092da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f8a0f6175764741bf7789b453d19e04', 'request_id': 'req-1e35a84d-73ff-4bf6-a95e-ab5427f092da'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:08.500417 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1e35a84d-73ff-4bf6-a95e-ab5427f092da admin admin] The token cache is being invalidate because project 2f8a0f6175764741bf7789b453d19e04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:21:08.500705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1e35a84d-73ff-4bf6-a95e-ab5427f092da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:08.501324 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-1e35a84d-73ff-4bf6-a95e-ab5427f092da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:08.502829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1356/2709] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:21:08 2022] DELETE /identity/v3/projects/2f8a0f6175764741bf7789b453d19e04 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:08.507035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f13821-aa44-4b0f-a7ef-f6a4b19f5d1e admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:08.508445 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-12f13821-aa44-4b0f-a7ef-f6a4b19f5d1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:08.521837 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f13821-aa44-4b0f-a7ef-f6a4b19f5d1e admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:08.578589 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d406a-c731-4787-8d47-3a2d90bc2341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:08.579371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d406a-c731-4787-8d47-3a2d90bc2341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:08.579654 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d406a-c731-4787-8d47-3a2d90bc2341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:08.579930 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d406a-c731-4787-8d47-3a2d90bc2341 admin admin] PATH_INFO: `/v3/users/6b7148c8084f47dcbe5d81cdac275462` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:08.581138 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d406a-c731-4787-8d47-3a2d90bc2341 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b7148c8084f47dcbe5d81cdac275462)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:08.583089 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d406a-c731-4787-8d47-3a2d90bc2341 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:08.630340 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-086d406a-c731-4787-8d47-3a2d90bc2341 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b7148c8084f47dcbe5d81cdac275462'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:08.637780 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-086d406a-c731-4787-8d47-3a2d90bc2341 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b7148c8084f47dcbe5d81cdac275462'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:08.644969 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-086d406a-c731-4787-8d47-3a2d90bc2341 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b7148c8084f47dcbe5d81cdac275462'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:08.645602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-086d406a-c731-4787-8d47-3a2d90bc2341 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b7148c8084f47dcbe5d81cdac275462'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:08.646708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:08.659433 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-086d406a-c731-4787-8d47-3a2d90bc2341 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b7148c8084f47dcbe5d81cdac275462'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:08.665274 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1354/2710] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:21:08 2022] DELETE /identity/v3/users/6b7148c8084f47dcbe5d81cdac275462 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:08.930138 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35a84d-73ff-4bf6-a95e-ab5427f092da admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:08.931676 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1e35a84d-73ff-4bf6-a95e-ab5427f092da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:08.956983 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35a84d-73ff-4bf6-a95e-ab5427f092da admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:09.015828 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4c56e-db6d-442d-87a5-f1f24d19430b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:09.016776 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4c56e-db6d-442d-87a5-f1f24d19430b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:09.017067 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4c56e-db6d-442d-87a5-f1f24d19430b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:09.017343 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4c56e-db6d-442d-87a5-f1f24d19430b admin admin] PATH_INFO: `/v3/projects/3a67f5d9b4b042ed929fa5c3d2e07276` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:09.018559 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4c56e-db6d-442d-87a5-f1f24d19430b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a67f5d9b4b042ed929fa5c3d2e07276)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:09.020584 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4c56e-db6d-442d-87a5-f1f24d19430b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:09.069904 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d6a4c56e-db6d-442d-87a5-f1f24d19430b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a67f5d9b4b042ed929fa5c3d2e07276', 'request_id': 'req-d6a4c56e-db6d-442d-87a5-f1f24d19430b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:09.076724 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d6a4c56e-db6d-442d-87a5-f1f24d19430b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a67f5d9b4b042ed929fa5c3d2e07276', 'request_id': 'req-d6a4c56e-db6d-442d-87a5-f1f24d19430b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:09.084163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d6a4c56e-db6d-442d-87a5-f1f24d19430b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a67f5d9b4b042ed929fa5c3d2e07276', 'request_id': 'req-d6a4c56e-db6d-442d-87a5-f1f24d19430b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:09.085411 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:09.097019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d6a4c56e-db6d-442d-87a5-f1f24d19430b admin admin] The token cache is being invalidate because project 3a67f5d9b4b042ed929fa5c3d2e07276 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:21:09.097312 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d6a4c56e-db6d-442d-87a5-f1f24d19430b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:09.097999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d6a4c56e-db6d-442d-87a5-f1f24d19430b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:09.099550 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1357/2711] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:21:08 2022] DELETE /identity/v3/projects/3a67f5d9b4b042ed929fa5c3d2e07276 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:12.872120 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d406a-c731-4787-8d47-3a2d90bc2341 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:12.873620 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-086d406a-c731-4787-8d47-3a2d90bc2341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:12.951241 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d406a-c731-4787-8d47-3a2d90bc2341 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:13.014039 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f38e37-5a2d-4768-8b69-33be9e7bd1b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:13.014616 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f38e37-5a2d-4768-8b69-33be9e7bd1b1 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:13.014904 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f38e37-5a2d-4768-8b69-33be9e7bd1b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:13.015187 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f38e37-5a2d-4768-8b69-33be9e7bd1b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:13.017030 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-84f38e37-5a2d-4768-8b69-33be9e7bd1b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:13.028607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f38e37-5a2d-4768-8b69-33be9e7bd1b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:13.030195 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f38e37-5a2d-4768-8b69-33be9e7bd1b1 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:13.040598 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1355/2712] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:21:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:21:16.296315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4c56e-db6d-442d-87a5-f1f24d19430b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:16.297715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d6a4c56e-db6d-442d-87a5-f1f24d19430b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:16.353883 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4c56e-db6d-442d-87a5-f1f24d19430b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:16.357955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc7311-55bb-4f42-960e-182053211857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:16.358608 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc7311-55bb-4f42-960e-182053211857 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:16.358897 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc7311-55bb-4f42-960e-182053211857 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:16.359180 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc7311-55bb-4f42-960e-182053211857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:16.360976 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b7dc7311-55bb-4f42-960e-182053211857 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:16.422374 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dc7311-55bb-4f42-960e-182053211857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:16.423814 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dc7311-55bb-4f42-960e-182053211857 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:16.434327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1358/2713] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:21:16 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:21:17.976535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f38e37-5a2d-4768-8b69-33be9e7bd1b1 service nova] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:17.977948 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-84f38e37-5a2d-4768-8b69-33be9e7bd1b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:17.997544 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f38e37-5a2d-4768-8b69-33be9e7bd1b1 service nova] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:18.003808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6873541f-9491-4b9a-9568-2df40e6279f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:18.004513 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873541f-9491-4b9a-9568-2df40e6279f9 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:18.004806 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873541f-9491-4b9a-9568-2df40e6279f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:18.005091 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873541f-9491-4b9a-9568-2df40e6279f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:18.006881 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6873541f-9491-4b9a-9568-2df40e6279f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:18.019477 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6873541f-9491-4b9a-9568-2df40e6279f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:18.021153 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6873541f-9491-4b9a-9568-2df40e6279f9 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:18.032517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1356/2714] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:21:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:21:26.429027 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=2,keystone:SELECT=53 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:28.027315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=75,keystone:DELETE=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:44.952101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d7298-3ee6-4275-9572-7d1aadce21ba None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:44.952601 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d7298-3ee6-4275-9572-7d1aadce21ba None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:44.952716 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d7298-3ee6-4275-9572-7d1aadce21ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:44.970168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2e3d7298-3ee6-4275-9572-7d1aadce21ba None None] MFA Rules not processed for user `8561df96cd7c47c89d283e36fc637bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:44.971382 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2e3d7298-3ee6-4275-9572-7d1aadce21ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:45.026296 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1359/2715] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:21:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:21:45.322918 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6873541f-9491-4b9a-9568-2df40e6279f9 service nova] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:45.324458 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6873541f-9491-4b9a-9568-2df40e6279f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:45.335830 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6873541f-9491-4b9a-9568-2df40e6279f9 service nova] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:45.339426 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c397c4-048a-42b5-865a-818df82260ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:45.340095 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c397c4-048a-42b5-865a-818df82260ed service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:45.340380 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c397c4-048a-42b5-865a-818df82260ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:45.340657 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c397c4-048a-42b5-865a-818df82260ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:45.342334 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-78c397c4-048a-42b5-865a-818df82260ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:45.351437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c397c4-048a-42b5-865a-818df82260ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:45.352878 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c397c4-048a-42b5-865a-818df82260ed service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:45.361184 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1357/2716] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:21:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:21:49.925097 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d7298-3ee6-4275-9572-7d1aadce21ba None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:49.943476 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d7298-3ee6-4275-9572-7d1aadce21ba None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:49.949812 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c2429-ae3d-43f4-955f-97028e0edbc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:49.950110 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c2429-ae3d-43f4-955f-97028e0edbc1 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:49.950386 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c2429-ae3d-43f4-955f-97028e0edbc1 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:49.950669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c2429-ae3d-43f4-955f-97028e0edbc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:49.952482 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d07c2429-ae3d-43f4-955f-97028e0edbc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:49.963327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07c2429-ae3d-43f4-955f-97028e0edbc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:49.964954 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07c2429-ae3d-43f4-955f-97028e0edbc1 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:49.974087 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1360/2717] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:21:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:21:55.357245 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=6 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:59.969817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=31 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:22:38.804688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c397c4-048a-42b5-865a-818df82260ed service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:38.806089 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-78c397c4-048a-42b5-865a-818df82260ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:38.873850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c397c4-048a-42b5-865a-818df82260ed service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:38.878764 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd160908-b5ce-4c27-9e72-378b7ca6f978 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd5b2ba9e37041c78a99f1700a4970c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:38.879484 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd160908-b5ce-4c27-9e72-378b7ca6f978 service placement] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:38.879774 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd160908-b5ce-4c27-9e72-378b7ca6f978 service placement] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:38.880088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd160908-b5ce-4c27-9e72-378b7ca6f978 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:38.881829 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-dd160908-b5ce-4c27-9e72-378b7ca6f978 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:38.893016 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd160908-b5ce-4c27-9e72-378b7ca6f978 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:38.895201 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd160908-b5ce-4c27-9e72-378b7ca6f978 service placement] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:38.905267 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1358/2718] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:22:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:22:48.901078 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=32 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:23:34.320050 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c2429-ae3d-43f4-955f-97028e0edbc1 service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:34.330227 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c2429-ae3d-43f4-955f-97028e0edbc1 service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:34.333959 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b5357b-d552-4698-a985-af75f765e132 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:34.334610 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5357b-d552-4698-a985-af75f765e132 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:34.334901 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5357b-d552-4698-a985-af75f765e132 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:34.335182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5357b-d552-4698-a985-af75f765e132 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:34.337046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a0b5357b-d552-4698-a985-af75f765e132 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:34.398761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b5357b-d552-4698-a985-af75f765e132 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:34.400242 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b5357b-d552-4698-a985-af75f765e132 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:34.409854 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1361/2719] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:23:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:23:34.667368 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd160908-b5ce-4c27-9e72-378b7ca6f978 service placement] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:34.676509 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd160908-b5ce-4c27-9e72-378b7ca6f978 service placement] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:34.680061 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783223f-c2e5-4898-8ebd-f2e1e61c0f37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:34.680700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783223f-c2e5-4898-8ebd-f2e1e61c0f37 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:34.680984 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783223f-c2e5-4898-8ebd-f2e1e61c0f37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:34.681961 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783223f-c2e5-4898-8ebd-f2e1e61c0f37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:34.683722 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2783223f-c2e5-4898-8ebd-f2e1e61c0f37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:34.697523 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783223f-c2e5-4898-8ebd-f2e1e61c0f37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:34.698955 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783223f-c2e5-4898-8ebd-f2e1e61c0f37 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:34.710309 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1359/2720] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:23:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:23:36.423465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0dc81-bc19-4025-bba8-6564c82b5b0f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:36.423990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0dc81-bc19-4025-bba8-6564c82b5b0f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:36.424234 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0dc81-bc19-4025-bba8-6564c82b5b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:36.446489 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6ac0dc81-bc19-4025-bba8-6564c82b5b0f None None] MFA Rules not processed for user `94d824bffc2c4af299c537056bc21734`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:23:36.448434 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6ac0dc81-bc19-4025-bba8-6564c82b5b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:36.513467 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1362/2721] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Jul 18 14:23:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:23:36.568814 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783223f-c2e5-4898-8ebd-f2e1e61c0f37 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:36.569882 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2783223f-c2e5-4898-8ebd-f2e1e61c0f37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:36.652548 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783223f-c2e5-4898-8ebd-f2e1e61c0f37 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:36.657521 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9b8ed-ebd1-4a52-b09e-28bf0427f304 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbf513ee25374e49b3d8ce28843e393a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:36.658176 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9b8ed-ebd1-4a52-b09e-28bf0427f304 service swift] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:36.658466 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9b8ed-ebd1-4a52-b09e-28bf0427f304 service swift] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:36.658745 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9b8ed-ebd1-4a52-b09e-28bf0427f304 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:36.668113 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be9b8ed-ebd1-4a52-b09e-28bf0427f304 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:36.669701 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be9b8ed-ebd1-4a52-b09e-28bf0427f304 service swift] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:36.677890 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1360/2722] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Jul 18 14:23:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:23:37.504607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0dc81-bc19-4025-bba8-6564c82b5b0f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:37.519750 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0dc81-bc19-4025-bba8-6564c82b5b0f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:37.526103 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a69f70-709f-48c6-a43a-f2839247d523 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:37.527595 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a69f70-709f-48c6-a43a-f2839247d523 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:37.528735 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a69f70-709f-48c6-a43a-f2839247d523 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:37.529176 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a69f70-709f-48c6-a43a-f2839247d523 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:37.532402 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-58a69f70-709f-48c6-a43a-f2839247d523 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:37.560269 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a69f70-709f-48c6-a43a-f2839247d523 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:37.562228 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a69f70-709f-48c6-a43a-f2839247d523 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:37.604254 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1363/2723] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:23:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:23:38.159919 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9b8ed-ebd1-4a52-b09e-28bf0427f304 service swift] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:38.162621 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6be9b8ed-ebd1-4a52-b09e-28bf0427f304 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:38.180548 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9b8ed-ebd1-4a52-b09e-28bf0427f304 service swift] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:38.193586 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04f002-4ec2-460b-ae77-85a9589e3f27 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd5b2ba9e37041c78a99f1700a4970c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:38.194539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04f002-4ec2-460b-ae77-85a9589e3f27 service placement] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:38.194992 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04f002-4ec2-460b-ae77-85a9589e3f27 service placement] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:38.195440 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04f002-4ec2-460b-ae77-85a9589e3f27 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:38.197972 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bc04f002-4ec2-460b-ae77-85a9589e3f27 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:38.219425 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc04f002-4ec2-460b-ae77-85a9589e3f27 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:38.222315 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc04f002-4ec2-460b-ae77-85a9589e3f27 service placement] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:38.239700 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1361/2724] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:23:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:23:42.227347 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a69f70-709f-48c6-a43a-f2839247d523 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:42.232062 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-58a69f70-709f-48c6-a43a-f2839247d523 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:42.247640 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a69f70-709f-48c6-a43a-f2839247d523 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:42.252391 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c4906-10b4-43bc-ae3e-438d247d0e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:42.253271 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c4906-10b4-43bc-ae3e-438d247d0e9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:42.253561 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c4906-10b4-43bc-ae3e-438d247d0e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:42.253844 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c4906-10b4-43bc-ae3e-438d247d0e9c admin admin] PATH_INFO: `/v3/users/35334a0f92254b5e900aad8c51873809` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:42.255222 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c4906-10b4-43bc-ae3e-438d247d0e9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=35334a0f92254b5e900aad8c51873809)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:42.257192 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c4906-10b4-43bc-ae3e-438d247d0e9c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:42.319382 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-728c4906-10b4-43bc-ae3e-438d247d0e9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35334a0f92254b5e900aad8c51873809'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:42.329034 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-728c4906-10b4-43bc-ae3e-438d247d0e9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35334a0f92254b5e900aad8c51873809'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:42.343343 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-728c4906-10b4-43bc-ae3e-438d247d0e9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35334a0f92254b5e900aad8c51873809'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:42.344115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-728c4906-10b4-43bc-ae3e-438d247d0e9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35334a0f92254b5e900aad8c51873809'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:42.360098 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-728c4906-10b4-43bc-ae3e-438d247d0e9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35334a0f92254b5e900aad8c51873809'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:42.361875 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:23:42.368006 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1364/2725] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:23:42 2022] DELETE /identity/v3/users/35334a0f92254b5e900aad8c51873809 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:23:42.568306 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04f002-4ec2-460b-ae77-85a9589e3f27 service placement] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:42.581921 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04f002-4ec2-460b-ae77-85a9589e3f27 service placement] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:42.664962 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587a1f2-e459-4dbd-bb2b-0da64c5be57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:42.665807 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587a1f2-e459-4dbd-bb2b-0da64c5be57a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:42.666306 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587a1f2-e459-4dbd-bb2b-0da64c5be57a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:42.666869 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587a1f2-e459-4dbd-bb2b-0da64c5be57a admin admin] PATH_INFO: `/v3/projects/9df5aea1d29c4a91b82a956e152bd214` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:42.668145 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587a1f2-e459-4dbd-bb2b-0da64c5be57a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9df5aea1d29c4a91b82a956e152bd214)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:42.670212 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587a1f2-e459-4dbd-bb2b-0da64c5be57a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:42.727859 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f587a1f2-e459-4dbd-bb2b-0da64c5be57a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9df5aea1d29c4a91b82a956e152bd214', 'request_id': 'req-f587a1f2-e459-4dbd-bb2b-0da64c5be57a'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:42.735132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f587a1f2-e459-4dbd-bb2b-0da64c5be57a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9df5aea1d29c4a91b82a956e152bd214', 'request_id': 'req-f587a1f2-e459-4dbd-bb2b-0da64c5be57a'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:42.746535 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f587a1f2-e459-4dbd-bb2b-0da64c5be57a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9df5aea1d29c4a91b82a956e152bd214', 'request_id': 'req-f587a1f2-e459-4dbd-bb2b-0da64c5be57a'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:42.748277 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:23:42.768562 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f587a1f2-e459-4dbd-bb2b-0da64c5be57a admin admin] The token cache is being invalidate because project 9df5aea1d29c4a91b82a956e152bd214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:23:42.768842 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f587a1f2-e459-4dbd-bb2b-0da64c5be57a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:42.769520 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f587a1f2-e459-4dbd-bb2b-0da64c5be57a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:42.771507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1362/2726] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:23:42 2022] DELETE /identity/v3/projects/9df5aea1d29c4a91b82a956e152bd214 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:23:42.777207 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c4906-10b4-43bc-ae3e-438d247d0e9c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:42.780533 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-728c4906-10b4-43bc-ae3e-438d247d0e9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:42.798937 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c4906-10b4-43bc-ae3e-438d247d0e9c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:42.874636 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf5e99-ebb3-48a4-a861-17dad76d7455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:42.875992 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf5e99-ebb3-48a4-a861-17dad76d7455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:42.876444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf5e99-ebb3-48a4-a861-17dad76d7455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:42.877451 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf5e99-ebb3-48a4-a861-17dad76d7455 admin admin] PATH_INFO: `/v3/users/7a5c06f51c99434d812700d710ebccaf` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:42.878945 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf5e99-ebb3-48a4-a861-17dad76d7455 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a5c06f51c99434d812700d710ebccaf)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:42.881241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf5e99-ebb3-48a4-a861-17dad76d7455 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:42.967052 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-31cf5e99-ebb3-48a4-a861-17dad76d7455 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a5c06f51c99434d812700d710ebccaf'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:42.976158 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-31cf5e99-ebb3-48a4-a861-17dad76d7455 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a5c06f51c99434d812700d710ebccaf'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:42.984706 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-31cf5e99-ebb3-48a4-a861-17dad76d7455 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a5c06f51c99434d812700d710ebccaf'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:42.985597 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-31cf5e99-ebb3-48a4-a861-17dad76d7455 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a5c06f51c99434d812700d710ebccaf'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:43.001590 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-31cf5e99-ebb3-48a4-a861-17dad76d7455 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a5c06f51c99434d812700d710ebccaf'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:43.008781 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1365/2727] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:23:42 2022] DELETE /identity/v3/users/7a5c06f51c99434d812700d710ebccaf => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:23:43.245271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587a1f2-e459-4dbd-bb2b-0da64c5be57a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:43.246873 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f587a1f2-e459-4dbd-bb2b-0da64c5be57a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:43.273495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587a1f2-e459-4dbd-bb2b-0da64c5be57a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:43.349690 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf95b1f-7eb9-404d-8c12-576c5c38bcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:43.350599 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf95b1f-7eb9-404d-8c12-576c5c38bcd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:43.350888 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf95b1f-7eb9-404d-8c12-576c5c38bcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:43.351194 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf95b1f-7eb9-404d-8c12-576c5c38bcd9 admin admin] PATH_INFO: `/v3/projects/9ce4b3331cb847c4a360c4da862dca29` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:43.352438 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf95b1f-7eb9-404d-8c12-576c5c38bcd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ce4b3331cb847c4a360c4da862dca29)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:43.354887 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf95b1f-7eb9-404d-8c12-576c5c38bcd9 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:43.408745 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-fdf95b1f-7eb9-404d-8c12-576c5c38bcd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ce4b3331cb847c4a360c4da862dca29', 'request_id': 'req-fdf95b1f-7eb9-404d-8c12-576c5c38bcd9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:43.414728 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-fdf95b1f-7eb9-404d-8c12-576c5c38bcd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ce4b3331cb847c4a360c4da862dca29', 'request_id': 'req-fdf95b1f-7eb9-404d-8c12-576c5c38bcd9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:43.429811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-fdf95b1f-7eb9-404d-8c12-576c5c38bcd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ce4b3331cb847c4a360c4da862dca29', 'request_id': 'req-fdf95b1f-7eb9-404d-8c12-576c5c38bcd9'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:43.445501 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-fdf95b1f-7eb9-404d-8c12-576c5c38bcd9 admin admin] The token cache is being invalidate because project 9ce4b3331cb847c4a360c4da862dca29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:23:43.445918 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-fdf95b1f-7eb9-404d-8c12-576c5c38bcd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:43.446660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-fdf95b1f-7eb9-404d-8c12-576c5c38bcd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:43.458823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1363/2728] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:23:43 2022] DELETE /identity/v3/projects/9ce4b3331cb847c4a360c4da862dca29 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:23:50.959981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932b068-20d9-41df-93f9-a8a3fd93f3b3 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:50.960667 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932b068-20d9-41df-93f9-a8a3fd93f3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:50.960911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932b068-20d9-41df-93f9-a8a3fd93f3b3 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:50.963716 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1366/2729] 10.222.0.94 () {54 vars in 921 bytes} [Mon Jul 18 14:23:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:23:50.969034 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa83486-ec31-4b21-9207-2b4280c5f341 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:50.969977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa83486-ec31-4b21-9207-2b4280c5f341 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:50.970264 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa83486-ec31-4b21-9207-2b4280c5f341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:50.990990 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-aaa83486-ec31-4b21-9207-2b4280c5f341 None None] MFA Rules not processed for user `ac717fcd9fc1499db12cf852d45572a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:23:50.993021 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-aaa83486-ec31-4b21-9207-2b4280c5f341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:51.088751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1364/2730] 10.222.0.94 () {58 vars in 1018 bytes} [Mon Jul 18 14:23:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:23:51.100323 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932b068-20d9-41df-93f9-a8a3fd93f3b3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:51.102482 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d932b068-20d9-41df-93f9-a8a3fd93f3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:51.188077 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932b068-20d9-41df-93f9-a8a3fd93f3b3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:51.191913 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c684acf-29ea-4617-837c-6f51273f3318 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '503a8228d4754c309fc3f365f4cfc61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:51.192591 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c684acf-29ea-4617-837c-6f51273f3318 service glance] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:51.192882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c684acf-29ea-4617-837c-6f51273f3318 service glance] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:51.193163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c684acf-29ea-4617-837c-6f51273f3318 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:51.204576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c684acf-29ea-4617-837c-6f51273f3318 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:51.206022 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c684acf-29ea-4617-837c-6f51273f3318 service glance] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:51.213605 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:23:51.219997 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1367/2731] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:23:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:23:51.298706 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa83486-ec31-4b21-9207-2b4280c5f341 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:51.300227 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-aaa83486-ec31-4b21-9207-2b4280c5f341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:51.318552 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:23:51.359789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa83486-ec31-4b21-9207-2b4280c5f341 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:51.365031 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1744b0-cdd5-4c95-a874-100b4e24706e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:51.366143 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1744b0-cdd5-4c95-a874-100b4e24706e service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:51.366655 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1744b0-cdd5-4c95-a874-100b4e24706e service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:51.367174 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1744b0-cdd5-4c95-a874-100b4e24706e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:51.370649 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ef1744b0-cdd5-4c95-a874-100b4e24706e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:51.387723 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1744b0-cdd5-4c95-a874-100b4e24706e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:51.389823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1744b0-cdd5-4c95-a874-100b4e24706e service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:51.397973 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1365/2732] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:23:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:23:51.472790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c684acf-29ea-4617-837c-6f51273f3318 service glance] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:51.482861 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c684acf-29ea-4617-837c-6f51273f3318 service glance] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:51.486372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49412ba-7425-49dd-a41a-c207ab41cf97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:51.487022 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49412ba-7425-49dd-a41a-c207ab41cf97 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:51.487350 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49412ba-7425-49dd-a41a-c207ab41cf97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:51.487639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49412ba-7425-49dd-a41a-c207ab41cf97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:51.489923 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f49412ba-7425-49dd-a41a-c207ab41cf97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:51.507152 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49412ba-7425-49dd-a41a-c207ab41cf97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:51.508888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49412ba-7425-49dd-a41a-c207ab41cf97 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:51.521224 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1368/2733] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:23:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:23:53.085464 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f9e6b-ccf6-406b-b9ed-05a5bf3dbc62 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:53.085934 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f9e6b-ccf6-406b-b9ed-05a5bf3dbc62 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:53.086041 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f9e6b-ccf6-406b-b9ed-05a5bf3dbc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:53.103845 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-7c4f9e6b-ccf6-406b-b9ed-05a5bf3dbc62 None None] MFA Rules not processed for user `94d824bffc2c4af299c537056bc21734`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:23:53.105084 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-7c4f9e6b-ccf6-406b-b9ed-05a5bf3dbc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:53.159778 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1366/2734] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Jul 18 14:23:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:23:53.171918 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49412ba-7425-49dd-a41a-c207ab41cf97 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:53.173333 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f49412ba-7425-49dd-a41a-c207ab41cf97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:53.240053 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49412ba-7425-49dd-a41a-c207ab41cf97 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:53.245605 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1592a3-89b4-4a39-b3ed-bcafc77c4ca9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbf513ee25374e49b3d8ce28843e393a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:53.246242 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1592a3-89b4-4a39-b3ed-bcafc77c4ca9 service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:53.246528 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1592a3-89b4-4a39-b3ed-bcafc77c4ca9 service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:53.246803 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1592a3-89b4-4a39-b3ed-bcafc77c4ca9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:53.254578 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1592a3-89b4-4a39-b3ed-bcafc77c4ca9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:53.256200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1592a3-89b4-4a39-b3ed-bcafc77c4ca9 service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:53.264114 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1369/2735] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Jul 18 14:23:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 14:24:03.153028 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=44 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:24:03.261022 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=38 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:24:11.629536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd0a11-11dc-424e-805a-5c687221bdd8 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:24:11.630010 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd0a11-11dc-424e-805a-5c687221bdd8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:24:11.630114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd0a11-11dc-424e-805a-5c687221bdd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:24:11.648068 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-82bd0a11-11dc-424e-805a-5c687221bdd8 None None] MFA Rules not processed for user `94d824bffc2c4af299c537056bc21734`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:24:11.649272 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-82bd0a11-11dc-424e-805a-5c687221bdd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:24:11.656403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1367/2736] 10.222.0.94 () {58 vars in 967 bytes} [Mon Jul 18 14:24:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:24:11.669216 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1592a3-89b4-4a39-b3ed-bcafc77c4ca9 service swift] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:24:11.679533 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1592a3-89b4-4a39-b3ed-bcafc77c4ca9 service swift] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:24:11.685197 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dea08-0df3-4abd-8803-6706a95f19d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbf513ee25374e49b3d8ce28843e393a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:24:11.685823 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dea08-0df3-4abd-8803-6706a95f19d3 service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:24:11.686111 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dea08-0df3-4abd-8803-6706a95f19d3 service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:24:11.686388 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dea08-0df3-4abd-8803-6706a95f19d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:24:11.694530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75dea08-0df3-4abd-8803-6706a95f19d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:24:11.696133 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75dea08-0df3-4abd-8803-6706a95f19d3 service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:24:11.703990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1370/2737] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Jul 18 14:24:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 14:24:21.642818 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:24:21.700798 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:24:44.140522 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd0a11-11dc-424e-805a-5c687221bdd8 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:24:44.142370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-82bd0a11-11dc-424e-805a-5c687221bdd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:24:44.153779 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd0a11-11dc-424e-805a-5c687221bdd8 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:24:44.157365 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92504e69-3125-4a36-8153-3fdf03d581e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:24:44.158011 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92504e69-3125-4a36-8153-3fdf03d581e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:24:44.158286 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92504e69-3125-4a36-8153-3fdf03d581e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:24:44.158562 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92504e69-3125-4a36-8153-3fdf03d581e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:24:44.160310 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-92504e69-3125-4a36-8153-3fdf03d581e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:24:44.169581 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92504e69-3125-4a36-8153-3fdf03d581e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:24:44.170986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92504e69-3125-4a36-8153-3fdf03d581e1 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:24:44.178981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1368/2738] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:24:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:24:54.175466 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=6 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:14.821759 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dea08-0df3-4abd-8803-6706a95f19d3 service swift] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:14.831620 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dea08-0df3-4abd-8803-6706a95f19d3 service swift] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:14.835192 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a2b8b-dd55-456d-97ab-6392fb31fa8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:14.835821 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a2b8b-dd55-456d-97ab-6392fb31fa8a service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:14.836142 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a2b8b-dd55-456d-97ab-6392fb31fa8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:14.836427 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a2b8b-dd55-456d-97ab-6392fb31fa8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:14.838188 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3a6a2b8b-dd55-456d-97ab-6392fb31fa8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:14.899817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a2b8b-dd55-456d-97ab-6392fb31fa8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:14.904694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a2b8b-dd55-456d-97ab-6392fb31fa8a service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:14.912669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1371/2739] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:25:17.921298 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92504e69-3125-4a36-8153-3fdf03d581e1 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:17.922962 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-92504e69-3125-4a36-8153-3fdf03d581e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:17.939327 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92504e69-3125-4a36-8153-3fdf03d581e1 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:17.947415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7441b5-be88-4d99-bff1-c183d697279e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:17.948270 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7441b5-be88-4d99-bff1-c183d697279e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:17.949058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7441b5-be88-4d99-bff1-c183d697279e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:17.949058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7441b5-be88-4d99-bff1-c183d697279e admin admin] PATH_INFO: `/v3/users/8561df96cd7c47c89d283e36fc637bb0` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:17.950035 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7441b5-be88-4d99-bff1-c183d697279e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8561df96cd7c47c89d283e36fc637bb0)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:17.952007 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7441b5-be88-4d99-bff1-c183d697279e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:18.007536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ce7441b5-be88-4d99-bff1-c183d697279e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8561df96cd7c47c89d283e36fc637bb0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:18.015304 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ce7441b5-be88-4d99-bff1-c183d697279e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8561df96cd7c47c89d283e36fc637bb0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:18.023446 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ce7441b5-be88-4d99-bff1-c183d697279e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8561df96cd7c47c89d283e36fc637bb0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:18.024162 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ce7441b5-be88-4d99-bff1-c183d697279e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8561df96cd7c47c89d283e36fc637bb0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:18.037775 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ce7441b5-be88-4d99-bff1-c183d697279e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8561df96cd7c47c89d283e36fc637bb0'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:18.044814 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1369/2740] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:25:17 2022] DELETE /identity/v3/users/8561df96cd7c47c89d283e36fc637bb0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:18.216458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a2b8b-dd55-456d-97ab-6392fb31fa8a service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:18.228740 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a2b8b-dd55-456d-97ab-6392fb31fa8a service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:18.287504 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152af4c3-0599-4fe2-b7dc-e23783e96edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:18.288407 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152af4c3-0599-4fe2-b7dc-e23783e96edb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:18.288699 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152af4c3-0599-4fe2-b7dc-e23783e96edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:18.288984 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152af4c3-0599-4fe2-b7dc-e23783e96edb admin admin] PATH_INFO: `/v3/projects/3e0348cc781c4af3bfc3302499b19c74` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:18.290163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152af4c3-0599-4fe2-b7dc-e23783e96edb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e0348cc781c4af3bfc3302499b19c74)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:18.292123 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152af4c3-0599-4fe2-b7dc-e23783e96edb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:18.338410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-152af4c3-0599-4fe2-b7dc-e23783e96edb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e0348cc781c4af3bfc3302499b19c74', 'request_id': 'req-152af4c3-0599-4fe2-b7dc-e23783e96edb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:18.344396 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-152af4c3-0599-4fe2-b7dc-e23783e96edb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e0348cc781c4af3bfc3302499b19c74', 'request_id': 'req-152af4c3-0599-4fe2-b7dc-e23783e96edb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:18.351898 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-152af4c3-0599-4fe2-b7dc-e23783e96edb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e0348cc781c4af3bfc3302499b19c74', 'request_id': 'req-152af4c3-0599-4fe2-b7dc-e23783e96edb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:18.362148 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-152af4c3-0599-4fe2-b7dc-e23783e96edb admin admin] The token cache is being invalidate because project 3e0348cc781c4af3bfc3302499b19c74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:25:18.362466 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-152af4c3-0599-4fe2-b7dc-e23783e96edb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:18.363098 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-152af4c3-0599-4fe2-b7dc-e23783e96edb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:18.364669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1372/2741] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:25:18 2022] DELETE /identity/v3/projects/3e0348cc781c4af3bfc3302499b19c74 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:18.369626 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59eb3b6-79a8-4f23-96b6-a5eca5e08c4b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:18.369949 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59eb3b6-79a8-4f23-96b6-a5eca5e08c4b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:18.370227 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59eb3b6-79a8-4f23-96b6-a5eca5e08c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:18.390771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-d59eb3b6-79a8-4f23-96b6-a5eca5e08c4b None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:18.392965 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d59eb3b6-79a8-4f23-96b6-a5eca5e08c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:18.458772 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1370/2742] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:18.466404 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34fa63-b038-4c01-98a6-02bda2303260 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:18.466637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34fa63-b038-4c01-98a6-02bda2303260 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:18.466913 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34fa63-b038-4c01-98a6-02bda2303260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:18.485421 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0c34fa63-b038-4c01-98a6-02bda2303260 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:18.486628 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0c34fa63-b038-4c01-98a6-02bda2303260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:18.494806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1373/2743] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:18.499037 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59eb3b6-79a8-4f23-96b6-a5eca5e08c4b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:18.510489 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59eb3b6-79a8-4f23-96b6-a5eca5e08c4b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:18.515098 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bf200-355d-4aea-8dd0-293b0c26a014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:18.515751 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bf200-355d-4aea-8dd0-293b0c26a014 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:18.516084 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bf200-355d-4aea-8dd0-293b0c26a014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:18.516371 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bf200-355d-4aea-8dd0-293b0c26a014 admin admin] PATH_INFO: `/v3/domains` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:18.517098 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7bf200-355d-4aea-8dd0-293b0c26a014 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80158) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:25:18.517445 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7bf200-355d-4aea-8dd0-293b0c26a014 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:18.519613 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7bf200-355d-4aea-8dd0-293b0c26a014 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:18.531348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1371/2744] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:25:18.535607 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34fa63-b038-4c01-98a6-02bda2303260 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:18.544968 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34fa63-b038-4c01-98a6-02bda2303260 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:18.549806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b8ce0-2df0-45f7-bd64-e41126a41011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:18.550453 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b8ce0-2df0-45f7-bd64-e41126a41011 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:18.550745 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b8ce0-2df0-45f7-bd64-e41126a41011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:18.551030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b8ce0-2df0-45f7-bd64-e41126a41011 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:18.551720 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2b8ce0-2df0-45f7-bd64-e41126a41011 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:18.553661 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2b8ce0-2df0-45f7-bd64-e41126a41011 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:18.568031 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:18.585786 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1374/2745] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:25:18 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:25:18.587064 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bf200-355d-4aea-8dd0-293b0c26a014 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:18.597118 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bf200-355d-4aea-8dd0-293b0c26a014 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:18.653203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcee32-2bb7-49e6-a82a-653210833a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:18.653864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcee32-2bb7-49e6-a82a-653210833a83 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:18.654150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcee32-2bb7-49e6-a82a-653210833a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:18.654428 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcee32-2bb7-49e6-a82a-653210833a83 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:18.655120 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fcee32-2bb7-49e6-a82a-653210833a83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:18.657094 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fcee32-2bb7-49e6-a82a-653210833a83 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:18.663611 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:18.677979 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1372/2746] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:25:18 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:25:18.682143 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b8ce0-2df0-45f7-bd64-e41126a41011 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:18.692930 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b8ce0-2df0-45f7-bd64-e41126a41011 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:18.697906 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285b4d8-fb61-417a-bd16-1f51b9251028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:18.698563 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285b4d8-fb61-417a-bd16-1f51b9251028 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:18.698853 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285b4d8-fb61-417a-bd16-1f51b9251028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:18.699135 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285b4d8-fb61-417a-bd16-1f51b9251028 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:18.699858 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285b4d8-fb61-417a-bd16-1f51b9251028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:18.701494 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285b4d8-fb61-417a-bd16-1f51b9251028 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:18.712500 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1375/2747] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:25:18.716141 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcee32-2bb7-49e6-a82a-653210833a83 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:18.726090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcee32-2bb7-49e6-a82a-653210833a83 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:18.730982 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec2a57-4683-4632-a00f-65bafd177f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:18.731642 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec2a57-4683-4632-a00f-65bafd177f13 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:18.732102 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec2a57-4683-4632-a00f-65bafd177f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:18.732311 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec2a57-4683-4632-a00f-65bafd177f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:18.733077 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec2a57-4683-4632-a00f-65bafd177f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:18.734674 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec2a57-4683-4632-a00f-65bafd177f13 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:18.744762 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1373/2748] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:25:18.748810 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285b4d8-fb61-417a-bd16-1f51b9251028 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:18.759225 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285b4d8-fb61-417a-bd16-1f51b9251028 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:18.763879 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208914f3-c643-4464-9202-af78b47150de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:18.764785 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208914f3-c643-4464-9202-af78b47150de admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:18.765078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208914f3-c643-4464-9202-af78b47150de admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:18.765364 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208914f3-c643-4464-9202-af78b47150de admin admin] PATH_INFO: `/v3/projects/b06671fbfa2142b4a7f60954465bc42e/users/e17367e1b9874be9b4b499ea232d2dd5/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:18.780463 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208914f3-c643-4464-9202-af78b47150de admin admin] RBAC: Authorizing `identity:create_grant(project_id=b06671fbfa2142b4a7f60954465bc42e, user_id=e17367e1b9874be9b4b499ea232d2dd5, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:18.783102 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208914f3-c643-4464-9202-af78b47150de admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:18.797423 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1376/2749] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:25:18 2022] PUT /identity/v3/projects/b06671fbfa2142b4a7f60954465bc42e/users/e17367e1b9874be9b4b499ea232d2dd5/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:18.808437 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4facb8-6751-49e3-8cc4-311a76ca81b8 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:18.808741 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4facb8-6751-49e3-8cc4-311a76ca81b8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:18.809055 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4facb8-6751-49e3-8cc4-311a76ca81b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:18.836629 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-ac4facb8-6751-49e3-8cc4-311a76ca81b8 None None] MFA Rules not processed for user `e17367e1b9874be9b4b499ea232d2dd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:18.837878 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-ac4facb8-6751-49e3-8cc4-311a76ca81b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:18.899681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1374/2750] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:25:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:18.904025 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208914f3-c643-4464-9202-af78b47150de admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:18.965620 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208914f3-c643-4464-9202-af78b47150de admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:18.970297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cd3a1-5e4a-41ab-8d15-843de7435d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:18.970955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cd3a1-5e4a-41ab-8d15-843de7435d48 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:18.971246 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cd3a1-5e4a-41ab-8d15-843de7435d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:18.971530 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cd3a1-5e4a-41ab-8d15-843de7435d48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:18.972284 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723cd3a1-5e4a-41ab-8d15-843de7435d48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:18.974195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723cd3a1-5e4a-41ab-8d15-843de7435d48 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:18.999990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1377/2751] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:25:18 2022] POST /identity/v3/projects => generated 369 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:25:19.004601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4facb8-6751-49e3-8cc4-311a76ca81b8 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:19.015259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4facb8-6751-49e3-8cc4-311a76ca81b8 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:19.071177 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffcfac-07f1-49f2-af85-0a160d52be38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:19.071853 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffcfac-07f1-49f2-af85-0a160d52be38 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:19.072171 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffcfac-07f1-49f2-af85-0a160d52be38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:19.072458 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffcfac-07f1-49f2-af85-0a160d52be38 admin admin] PATH_INFO: `/v3/users` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:19.073192 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ffcfac-07f1-49f2-af85-0a160d52be38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:19.075113 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ffcfac-07f1-49f2-af85-0a160d52be38 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:19.094194 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1375/2752] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:25:19 2022] POST /identity/v3/users => generated 291 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:25:19.098194 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cd3a1-5e4a-41ab-8d15-843de7435d48 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:19.108322 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cd3a1-5e4a-41ab-8d15-843de7435d48 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:19.113461 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad87d6-2fed-47ea-ad9b-aa1a2e8e97ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:19.114108 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad87d6-2fed-47ea-ad9b-aa1a2e8e97ce admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:19.114391 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad87d6-2fed-47ea-ad9b-aa1a2e8e97ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:19.114670 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad87d6-2fed-47ea-ad9b-aa1a2e8e97ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:19.115398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad87d6-2fed-47ea-ad9b-aa1a2e8e97ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:19.116993 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad87d6-2fed-47ea-ad9b-aa1a2e8e97ce admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:19.127261 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1378/2753] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:25:19.131825 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffcfac-07f1-49f2-af85-0a160d52be38 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:19.141870 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffcfac-07f1-49f2-af85-0a160d52be38 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:19.146722 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99d3ea-ef41-4f43-bb87-0e723738fea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:19.147594 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99d3ea-ef41-4f43-bb87-0e723738fea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:19.147876 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99d3ea-ef41-4f43-bb87-0e723738fea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:19.148184 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99d3ea-ef41-4f43-bb87-0e723738fea3 admin admin] PATH_INFO: `/v3/projects/56484b44a36b46878b28a76c70ed91ab/users/673519bd1d7e4a399348cc0c1c4ffa90/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:19.162670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99d3ea-ef41-4f43-bb87-0e723738fea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56484b44a36b46878b28a76c70ed91ab, user_id=673519bd1d7e4a399348cc0c1c4ffa90, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:19.165381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99d3ea-ef41-4f43-bb87-0e723738fea3 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:19.178252 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1376/2754] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:25:19 2022] PUT /identity/v3/projects/56484b44a36b46878b28a76c70ed91ab/users/673519bd1d7e4a399348cc0c1c4ffa90/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:19.182078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad87d6-2fed-47ea-ad9b-aa1a2e8e97ce admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:19.192960 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad87d6-2fed-47ea-ad9b-aa1a2e8e97ce admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:19.248970 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b121e1-07ef-44a6-ba20-bb0a6621ebf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:19.249634 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b121e1-07ef-44a6-ba20-bb0a6621ebf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:19.249926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b121e1-07ef-44a6-ba20-bb0a6621ebf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:19.250259 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b121e1-07ef-44a6-ba20-bb0a6621ebf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:19.250933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b121e1-07ef-44a6-ba20-bb0a6621ebf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:19.252572 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b121e1-07ef-44a6-ba20-bb0a6621ebf4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:19.262182 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1379/2755] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:25:19.266294 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99d3ea-ef41-4f43-bb87-0e723738fea3 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:19.276544 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99d3ea-ef41-4f43-bb87-0e723738fea3 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:19.281114 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304003a7-670b-4d1c-84ec-4fb0429623c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:19.282006 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304003a7-670b-4d1c-84ec-4fb0429623c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:19.282293 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304003a7-670b-4d1c-84ec-4fb0429623c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:19.282572 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304003a7-670b-4d1c-84ec-4fb0429623c8 admin admin] PATH_INFO: `/v3/projects/56484b44a36b46878b28a76c70ed91ab/users/673519bd1d7e4a399348cc0c1c4ffa90/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:19.285132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304003a7-670b-4d1c-84ec-4fb0429623c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56484b44a36b46878b28a76c70ed91ab, user_id=673519bd1d7e4a399348cc0c1c4ffa90, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:19.287724 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304003a7-670b-4d1c-84ec-4fb0429623c8 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:19.297685 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1377/2756] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:25:19 2022] PUT /identity/v3/projects/56484b44a36b46878b28a76c70ed91ab/users/673519bd1d7e4a399348cc0c1c4ffa90/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:19.308144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2f064-1ff5-4ea4-a6e2-9bd108ed7083 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:19.308443 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2f064-1ff5-4ea4-a6e2-9bd108ed7083 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:19.308746 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2f064-1ff5-4ea4-a6e2-9bd108ed7083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:19.334668 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-26f2f064-1ff5-4ea4-a6e2-9bd108ed7083 None None] MFA Rules not processed for user `673519bd1d7e4a399348cc0c1c4ffa90`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:19.335884 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-26f2f064-1ff5-4ea4-a6e2-9bd108ed7083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:19.354760 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:19.403705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1380/2757] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:25:19 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:19.415273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304003a7-670b-4d1c-84ec-4fb0429623c8 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:19.417001 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-304003a7-670b-4d1c-84ec-4fb0429623c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:19.439543 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:19.474714 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304003a7-670b-4d1c-84ec-4fb0429623c8 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:19.478387 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c773c2b-a06b-460d-b40c-da85468576a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:19.479031 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c773c2b-a06b-460d-b40c-da85468576a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:19.479320 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c773c2b-a06b-460d-b40c-da85468576a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:19.479601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c773c2b-a06b-460d-b40c-da85468576a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:19.489030 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c773c2b-a06b-460d-b40c-da85468576a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:19.490482 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c773c2b-a06b-460d-b40c-da85468576a8 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:19.499828 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1378/2758] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:19 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:25:19.865444 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2f064-1ff5-4ea4-a6e2-9bd108ed7083 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:19.926381 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2f064-1ff5-4ea4-a6e2-9bd108ed7083 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:19.930890 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60722613-7922-4749-9462-ef8d0bc9189e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:19.931746 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60722613-7922-4749-9462-ef8d0bc9189e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:19.932065 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60722613-7922-4749-9462-ef8d0bc9189e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:19.932345 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60722613-7922-4749-9462-ef8d0bc9189e admin admin] PATH_INFO: `/v3/users/e17367e1b9874be9b4b499ea232d2dd5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:19.933522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60722613-7922-4749-9462-ef8d0bc9189e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e17367e1b9874be9b4b499ea232d2dd5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:19.935427 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60722613-7922-4749-9462-ef8d0bc9189e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:19.984040 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-60722613-7922-4749-9462-ef8d0bc9189e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e17367e1b9874be9b4b499ea232d2dd5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:19.991660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-60722613-7922-4749-9462-ef8d0bc9189e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e17367e1b9874be9b4b499ea232d2dd5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:19.998562 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-60722613-7922-4749-9462-ef8d0bc9189e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e17367e1b9874be9b4b499ea232d2dd5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:19.999195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-60722613-7922-4749-9462-ef8d0bc9189e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e17367e1b9874be9b4b499ea232d2dd5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:20.011016 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-60722613-7922-4749-9462-ef8d0bc9189e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e17367e1b9874be9b4b499ea232d2dd5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:20.016956 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1381/2759] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:25:19 2022] DELETE /identity/v3/users/e17367e1b9874be9b4b499ea232d2dd5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:20.021393 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d72a6b-d97e-4819-a7c7-211370327370 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:20.021868 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d72a6b-d97e-4819-a7c7-211370327370 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:20.021984 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d72a6b-d97e-4819-a7c7-211370327370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:20.039980 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-e1d72a6b-d97e-4819-a7c7-211370327370 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:20.041190 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e1d72a6b-d97e-4819-a7c7-211370327370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:20.099986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1379/2760] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:20.109535 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60722613-7922-4749-9462-ef8d0bc9189e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:20.110915 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-60722613-7922-4749-9462-ef8d0bc9189e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:20.151669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:20.170502 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60722613-7922-4749-9462-ef8d0bc9189e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:20.174122 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a9f6d-4afb-48ff-88e4-9a5806655270 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:20.175404 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a9f6d-4afb-48ff-88e4-9a5806655270 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:20.175694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a9f6d-4afb-48ff-88e4-9a5806655270 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:20.176003 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a9f6d-4afb-48ff-88e4-9a5806655270 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:20.184780 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a9f6d-4afb-48ff-88e4-9a5806655270 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:20.186199 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a9f6d-4afb-48ff-88e4-9a5806655270 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:20.195767 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1382/2761] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:25:20.463076 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d72a6b-d97e-4819-a7c7-211370327370 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:20.464532 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e1d72a6b-d97e-4819-a7c7-211370327370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:20.478210 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d72a6b-d97e-4819-a7c7-211370327370 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:20.483383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a393044-357b-4859-8ac1-6fe6b1fa9440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:20.484283 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a393044-357b-4859-8ac1-6fe6b1fa9440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:20.484571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a393044-357b-4859-8ac1-6fe6b1fa9440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:20.484847 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a393044-357b-4859-8ac1-6fe6b1fa9440 admin admin] PATH_INFO: `/v3/projects/b06671fbfa2142b4a7f60954465bc42e` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:20.486034 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a393044-357b-4859-8ac1-6fe6b1fa9440 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b06671fbfa2142b4a7f60954465bc42e)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:20.488723 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a393044-357b-4859-8ac1-6fe6b1fa9440 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:20.537898 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0a393044-357b-4859-8ac1-6fe6b1fa9440 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b06671fbfa2142b4a7f60954465bc42e', 'request_id': 'req-0a393044-357b-4859-8ac1-6fe6b1fa9440'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:20.543239 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0a393044-357b-4859-8ac1-6fe6b1fa9440 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b06671fbfa2142b4a7f60954465bc42e', 'request_id': 'req-0a393044-357b-4859-8ac1-6fe6b1fa9440'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:20.550469 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0a393044-357b-4859-8ac1-6fe6b1fa9440 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b06671fbfa2142b4a7f60954465bc42e', 'request_id': 'req-0a393044-357b-4859-8ac1-6fe6b1fa9440'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:20.561464 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0a393044-357b-4859-8ac1-6fe6b1fa9440 admin admin] The token cache is being invalidate because project b06671fbfa2142b4a7f60954465bc42e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:25:20.561756 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0a393044-357b-4859-8ac1-6fe6b1fa9440 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:20.562382 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-0a393044-357b-4859-8ac1-6fe6b1fa9440 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:20.563937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1380/2762] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:25:20 2022] DELETE /identity/v3/projects/b06671fbfa2142b4a7f60954465bc42e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:20.567921 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a9f6d-4afb-48ff-88e4-9a5806655270 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:20.577890 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a9f6d-4afb-48ff-88e4-9a5806655270 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:20.633801 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab3a50-ad37-4bc5-b70a-6502cd882a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:20.634673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab3a50-ad37-4bc5-b70a-6502cd882a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:20.634966 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab3a50-ad37-4bc5-b70a-6502cd882a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:20.635251 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab3a50-ad37-4bc5-b70a-6502cd882a6f admin admin] PATH_INFO: `/v3/users/673519bd1d7e4a399348cc0c1c4ffa90` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:20.636465 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab3a50-ad37-4bc5-b70a-6502cd882a6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=673519bd1d7e4a399348cc0c1c4ffa90)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:20.638389 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab3a50-ad37-4bc5-b70a-6502cd882a6f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:20.682664 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d4ab3a50-ad37-4bc5-b70a-6502cd882a6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '673519bd1d7e4a399348cc0c1c4ffa90'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:20.689666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d4ab3a50-ad37-4bc5-b70a-6502cd882a6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '673519bd1d7e4a399348cc0c1c4ffa90'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:20.696921 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d4ab3a50-ad37-4bc5-b70a-6502cd882a6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '673519bd1d7e4a399348cc0c1c4ffa90'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:20.697583 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d4ab3a50-ad37-4bc5-b70a-6502cd882a6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '673519bd1d7e4a399348cc0c1c4ffa90'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:20.709937 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d4ab3a50-ad37-4bc5-b70a-6502cd882a6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '673519bd1d7e4a399348cc0c1c4ffa90'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:20.715821 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1383/2763] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:25:20 2022] DELETE /identity/v3/users/673519bd1d7e4a399348cc0c1c4ffa90 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:20.986634 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a393044-357b-4859-8ac1-6fe6b1fa9440 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:20.988126 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-0a393044-357b-4859-8ac1-6fe6b1fa9440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:21.002266 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a393044-357b-4859-8ac1-6fe6b1fa9440 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:21.047418 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:21.061571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a14ecf-8027-42e5-87b5-bfececafb7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:21.062461 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a14ecf-8027-42e5-87b5-bfececafb7a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:21.062749 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a14ecf-8027-42e5-87b5-bfececafb7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:21.063029 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a14ecf-8027-42e5-87b5-bfececafb7a4 admin admin] PATH_INFO: `/v3/projects/56484b44a36b46878b28a76c70ed91ab` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:21.064259 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a14ecf-8027-42e5-87b5-bfececafb7a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56484b44a36b46878b28a76c70ed91ab)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:21.066657 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a14ecf-8027-42e5-87b5-bfececafb7a4 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:21.111268 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a2a14ecf-8027-42e5-87b5-bfececafb7a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56484b44a36b46878b28a76c70ed91ab', 'request_id': 'req-a2a14ecf-8027-42e5-87b5-bfececafb7a4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:21.117167 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a2a14ecf-8027-42e5-87b5-bfececafb7a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56484b44a36b46878b28a76c70ed91ab', 'request_id': 'req-a2a14ecf-8027-42e5-87b5-bfececafb7a4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:21.124139 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a2a14ecf-8027-42e5-87b5-bfececafb7a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56484b44a36b46878b28a76c70ed91ab', 'request_id': 'req-a2a14ecf-8027-42e5-87b5-bfececafb7a4'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:21.135553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a2a14ecf-8027-42e5-87b5-bfececafb7a4 admin admin] The token cache is being invalidate because project 56484b44a36b46878b28a76c70ed91ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:25:21.135847 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a2a14ecf-8027-42e5-87b5-bfececafb7a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:21.136520 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-a2a14ecf-8027-42e5-87b5-bfececafb7a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:21.138140 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1381/2764] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:25:20 2022] DELETE /identity/v3/projects/56484b44a36b46878b28a76c70ed91ab => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:21.143048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d9d1d-05b7-4881-a5b7-a5ce7a49773e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:21.143365 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d9d1d-05b7-4881-a5b7-a5ce7a49773e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:21.143647 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d9d1d-05b7-4881-a5b7-a5ce7a49773e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:21.160847 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-135d9d1d-05b7-4881-a5b7-a5ce7a49773e None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:21.162049 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-135d9d1d-05b7-4881-a5b7-a5ce7a49773e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:21.211204 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:21.222151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1384/2765] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:21.230349 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4dd7d-084e-4b1d-a12f-0ccf76f3753c None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:21.230669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4dd7d-084e-4b1d-a12f-0ccf76f3753c None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:21.230999 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4dd7d-084e-4b1d-a12f-0ccf76f3753c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:21.249403 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-3bb4dd7d-084e-4b1d-a12f-0ccf76f3753c None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:21.250634 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3bb4dd7d-084e-4b1d-a12f-0ccf76f3753c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:21.259263 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1382/2766] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:21.263882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d9d1d-05b7-4881-a5b7-a5ce7a49773e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:21.274057 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d9d1d-05b7-4881-a5b7-a5ce7a49773e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:21.278690 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd02e1-8e2a-4017-9787-afcdef7d410f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:21.279339 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd02e1-8e2a-4017-9787-afcdef7d410f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:21.279626 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd02e1-8e2a-4017-9787-afcdef7d410f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:21.279926 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd02e1-8e2a-4017-9787-afcdef7d410f admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:21.280679 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd02e1-8e2a-4017-9787-afcdef7d410f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:25:21.280985 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd02e1-8e2a-4017-9787-afcdef7d410f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:21.282569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd02e1-8e2a-4017-9787-afcdef7d410f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:21.294760 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1385/2767] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:25:21.299215 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4dd7d-084e-4b1d-a12f-0ccf76f3753c None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:21.309213 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4dd7d-084e-4b1d-a12f-0ccf76f3753c None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:21.313993 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0805fa-292f-4633-b5ce-88f53d8eadde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:21.314646 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0805fa-292f-4633-b5ce-88f53d8eadde admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:21.314938 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0805fa-292f-4633-b5ce-88f53d8eadde admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:21.315244 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0805fa-292f-4633-b5ce-88f53d8eadde admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:21.315953 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0805fa-292f-4633-b5ce-88f53d8eadde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:21.318473 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0805fa-292f-4633-b5ce-88f53d8eadde admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:21.343548 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1383/2768] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:25:21 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:25:21.347440 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd02e1-8e2a-4017-9787-afcdef7d410f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:21.357277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd02e1-8e2a-4017-9787-afcdef7d410f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:21.412857 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d0605-011f-450e-a459-675c3d13ba31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:21.413523 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d0605-011f-450e-a459-675c3d13ba31 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:21.413845 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d0605-011f-450e-a459-675c3d13ba31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:21.414134 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d0605-011f-450e-a459-675c3d13ba31 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:21.414841 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d0605-011f-450e-a459-675c3d13ba31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:21.416787 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d0605-011f-450e-a459-675c3d13ba31 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:21.436612 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1386/2769] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:25:21 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:25:21.440667 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0805fa-292f-4633-b5ce-88f53d8eadde admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:21.451786 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0805fa-292f-4633-b5ce-88f53d8eadde admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:21.456436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238638b5-89d6-4709-a9d6-b2e3b346ce4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:21.457091 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238638b5-89d6-4709-a9d6-b2e3b346ce4b admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:21.457382 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238638b5-89d6-4709-a9d6-b2e3b346ce4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:21.457666 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238638b5-89d6-4709-a9d6-b2e3b346ce4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:21.458398 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238638b5-89d6-4709-a9d6-b2e3b346ce4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:21.460053 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238638b5-89d6-4709-a9d6-b2e3b346ce4b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:21.470216 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1384/2770] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:25:21.474365 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d0605-011f-450e-a459-675c3d13ba31 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:21.484179 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d0605-011f-450e-a459-675c3d13ba31 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:21.489466 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329658a-0ea0-4769-95ab-ae2b435581b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:21.490128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329658a-0ea0-4769-95ab-ae2b435581b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:21.490420 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329658a-0ea0-4769-95ab-ae2b435581b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:21.490730 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329658a-0ea0-4769-95ab-ae2b435581b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:21.491455 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329658a-0ea0-4769-95ab-ae2b435581b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:21.493145 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329658a-0ea0-4769-95ab-ae2b435581b2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:21.503092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1387/2771] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:25:21.506836 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238638b5-89d6-4709-a9d6-b2e3b346ce4b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:21.516565 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238638b5-89d6-4709-a9d6-b2e3b346ce4b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:21.521635 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff296ce4-ab37-4ebd-826a-c12db397f29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:21.522690 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff296ce4-ab37-4ebd-826a-c12db397f29d admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:21.523127 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff296ce4-ab37-4ebd-826a-c12db397f29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:21.523264 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff296ce4-ab37-4ebd-826a-c12db397f29d admin admin] PATH_INFO: `/v3/projects/eb0f79114b594ef5928b9be51715163e/users/61611b3d7ad04f15801a1db20c2e1773/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:21.539080 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff296ce4-ab37-4ebd-826a-c12db397f29d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb0f79114b594ef5928b9be51715163e, user_id=61611b3d7ad04f15801a1db20c2e1773, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:21.541981 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff296ce4-ab37-4ebd-826a-c12db397f29d admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:21.555359 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1385/2772] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:25:21 2022] PUT /identity/v3/projects/eb0f79114b594ef5928b9be51715163e/users/61611b3d7ad04f15801a1db20c2e1773/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:25:21.566860 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7f9eb-de17-46df-a827-90ba0a1ddd1b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:21.567153 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7f9eb-de17-46df-a827-90ba0a1ddd1b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:21.567436 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7f9eb-de17-46df-a827-90ba0a1ddd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:21.593903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-83e7f9eb-de17-46df-a827-90ba0a1ddd1b None None] MFA Rules not processed for user `61611b3d7ad04f15801a1db20c2e1773`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:21.595118 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-83e7f9eb-de17-46df-a827-90ba0a1ddd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:21.658103 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1388/2773] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:25:21 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:21.663127 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff296ce4-ab37-4ebd-826a-c12db397f29d admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:21.723417 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff296ce4-ab37-4ebd-826a-c12db397f29d admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:21.728532 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff7564-4863-46bc-b5ac-00100b2e83b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:21.729183 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff7564-4863-46bc-b5ac-00100b2e83b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:21.729472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff7564-4863-46bc-b5ac-00100b2e83b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:21.729773 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff7564-4863-46bc-b5ac-00100b2e83b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:21.730483 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff7564-4863-46bc-b5ac-00100b2e83b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:21.732405 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff7564-4863-46bc-b5ac-00100b2e83b1 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:21.750531 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:21.758188 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1386/2774] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:25:21 2022] POST /identity/v3/projects => generated 363 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:25:21.761999 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7f9eb-de17-46df-a827-90ba0a1ddd1b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:21.772491 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7f9eb-de17-46df-a827-90ba0a1ddd1b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:21.820601 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:21.830241 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37f779-bfa3-4c0c-8d50-8a535a50e42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:21.830908 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37f779-bfa3-4c0c-8d50-8a535a50e42f admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:21.831198 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37f779-bfa3-4c0c-8d50-8a535a50e42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:21.831484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37f779-bfa3-4c0c-8d50-8a535a50e42f admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:21.832207 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37f779-bfa3-4c0c-8d50-8a535a50e42f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:21.834614 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37f779-bfa3-4c0c-8d50-8a535a50e42f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:21.854104 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1389/2775] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:25:21 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:25:21.858078 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff7564-4863-46bc-b5ac-00100b2e83b1 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:21.868895 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff7564-4863-46bc-b5ac-00100b2e83b1 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:21.873608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3a86a-f653-4615-b278-8b9f3519d42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:21.874271 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3a86a-f653-4615-b278-8b9f3519d42f admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:21.874562 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3a86a-f653-4615-b278-8b9f3519d42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:21.874849 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3a86a-f653-4615-b278-8b9f3519d42f admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:21.875576 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3a86a-f653-4615-b278-8b9f3519d42f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:21.877213 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3a86a-f653-4615-b278-8b9f3519d42f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:21.887591 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1387/2776] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:25:21.891573 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37f779-bfa3-4c0c-8d50-8a535a50e42f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:21.902025 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37f779-bfa3-4c0c-8d50-8a535a50e42f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:21.906675 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bab2d-2417-42f7-b130-8f63e60c530e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:21.907552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bab2d-2417-42f7-b130-8f63e60c530e admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:21.907842 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bab2d-2417-42f7-b130-8f63e60c530e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:21.908151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bab2d-2417-42f7-b130-8f63e60c530e admin admin] PATH_INFO: `/v3/projects/b4df57389fbe4f4382598e76184f410c/users/0efa165f910d4fa1ad72e7f01b2317d2/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:21.923081 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910bab2d-2417-42f7-b130-8f63e60c530e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4df57389fbe4f4382598e76184f410c, user_id=0efa165f910d4fa1ad72e7f01b2317d2, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:21.925715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910bab2d-2417-42f7-b130-8f63e60c530e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:21.938352 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1390/2777] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:25:21 2022] PUT /identity/v3/projects/b4df57389fbe4f4382598e76184f410c/users/0efa165f910d4fa1ad72e7f01b2317d2/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:25:21.942388 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3a86a-f653-4615-b278-8b9f3519d42f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:21.952774 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3a86a-f653-4615-b278-8b9f3519d42f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:22.007917 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878f170-e02d-4c8f-9e49-cb37bea172ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:22.008604 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878f170-e02d-4c8f-9e49-cb37bea172ea admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:22.008895 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878f170-e02d-4c8f-9e49-cb37bea172ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:22.009180 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878f170-e02d-4c8f-9e49-cb37bea172ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:22.009895 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878f170-e02d-4c8f-9e49-cb37bea172ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:22.011498 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878f170-e02d-4c8f-9e49-cb37bea172ea admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:22.021578 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1388/2778] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:25:22.025668 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bab2d-2417-42f7-b130-8f63e60c530e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:22.035834 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bab2d-2417-42f7-b130-8f63e60c530e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:22.041127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20112a-315a-4c58-9e20-d998e1667332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:22.042003 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20112a-315a-4c58-9e20-d998e1667332 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:22.042294 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20112a-315a-4c58-9e20-d998e1667332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:22.042575 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20112a-315a-4c58-9e20-d998e1667332 admin admin] PATH_INFO: `/v3/projects/b4df57389fbe4f4382598e76184f410c/users/0efa165f910d4fa1ad72e7f01b2317d2/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:22.044580 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20112a-315a-4c58-9e20-d998e1667332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4df57389fbe4f4382598e76184f410c, user_id=0efa165f910d4fa1ad72e7f01b2317d2, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:22.047209 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20112a-315a-4c58-9e20-d998e1667332 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:22.057288 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1391/2779] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:25:22 2022] PUT /identity/v3/projects/b4df57389fbe4f4382598e76184f410c/users/0efa165f910d4fa1ad72e7f01b2317d2/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:22.067118 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd1b9f-075e-4d6f-ae64-57067d8b5f18 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:22.067599 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd1b9f-075e-4d6f-ae64-57067d8b5f18 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:22.067701 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd1b9f-075e-4d6f-ae64-57067d8b5f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:22.094282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-5bdd1b9f-075e-4d6f-ae64-57067d8b5f18 None None] MFA Rules not processed for user `0efa165f910d4fa1ad72e7f01b2317d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:22.095498 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-5bdd1b9f-075e-4d6f-ae64-57067d8b5f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:22.162517 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1389/2780] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:25:22 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:22.173603 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20112a-315a-4c58-9e20-d998e1667332 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:22.175230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9f20112a-315a-4c58-9e20-d998e1667332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:22.232319 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20112a-315a-4c58-9e20-d998e1667332 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:22.236020 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18eb1e-c356-41c2-9003-79a277ea9e1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:22.236669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18eb1e-c356-41c2-9003-79a277ea9e1b service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:22.236960 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18eb1e-c356-41c2-9003-79a277ea9e1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:22.237246 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18eb1e-c356-41c2-9003-79a277ea9e1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:22.246357 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b18eb1e-c356-41c2-9003-79a277ea9e1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:22.247797 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b18eb1e-c356-41c2-9003-79a277ea9e1b service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:22.257745 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1392/2781] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:22 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:25:22.327984 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd1b9f-075e-4d6f-ae64-57067d8b5f18 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:22.338582 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd1b9f-075e-4d6f-ae64-57067d8b5f18 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:22.342153 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8555f80-7e9a-48ce-a85e-3a05c69b89fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:22.342781 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8555f80-7e9a-48ce-a85e-3a05c69b89fb service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:22.343064 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8555f80-7e9a-48ce-a85e-3a05c69b89fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:22.343337 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8555f80-7e9a-48ce-a85e-3a05c69b89fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:22.399381 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8555f80-7e9a-48ce-a85e-3a05c69b89fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:22.400833 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8555f80-7e9a-48ce-a85e-3a05c69b89fb service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:22.409544 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1390/2782] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:22 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:25:32.253003 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=46,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:32.405483 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:33.795648 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18eb1e-c356-41c2-9003-79a277ea9e1b service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:33.858422 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18eb1e-c356-41c2-9003-79a277ea9e1b service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:33.863061 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f8b63-7c76-4118-8f0f-f1da10a21d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:33.863921 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f8b63-7c76-4118-8f0f-f1da10a21d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:33.864247 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f8b63-7c76-4118-8f0f-f1da10a21d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:33.864526 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f8b63-7c76-4118-8f0f-f1da10a21d1b admin admin] PATH_INFO: `/v3/users/61611b3d7ad04f15801a1db20c2e1773` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:33.866046 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f8b63-7c76-4118-8f0f-f1da10a21d1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=61611b3d7ad04f15801a1db20c2e1773)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:33.868014 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f8b63-7c76-4118-8f0f-f1da10a21d1b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:33.914306 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-da9f8b63-7c76-4118-8f0f-f1da10a21d1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61611b3d7ad04f15801a1db20c2e1773'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:33.921694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-da9f8b63-7c76-4118-8f0f-f1da10a21d1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61611b3d7ad04f15801a1db20c2e1773'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:33.928735 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-da9f8b63-7c76-4118-8f0f-f1da10a21d1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61611b3d7ad04f15801a1db20c2e1773'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:33.929425 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-da9f8b63-7c76-4118-8f0f-f1da10a21d1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61611b3d7ad04f15801a1db20c2e1773'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:33.941174 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-da9f8b63-7c76-4118-8f0f-f1da10a21d1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61611b3d7ad04f15801a1db20c2e1773'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:33.947891 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1393/2783] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:25:33 2022] DELETE /identity/v3/users/61611b3d7ad04f15801a1db20c2e1773 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:33.952415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf6e20-7446-4529-9a5c-e2c45e34da8f None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:33.952882 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf6e20-7446-4529-9a5c-e2c45e34da8f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:33.952988 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf6e20-7446-4529-9a5c-e2c45e34da8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:33.970164 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-edaf6e20-7446-4529-9a5c-e2c45e34da8f None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:33.971478 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-edaf6e20-7446-4529-9a5c-e2c45e34da8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:34.030313 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1391/2784] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:34.040922 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f8b63-7c76-4118-8f0f-f1da10a21d1b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:34.042321 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-da9f8b63-7c76-4118-8f0f-f1da10a21d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:34.099749 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f8b63-7c76-4118-8f0f-f1da10a21d1b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:34.103487 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855206f2-a0c6-4d74-ae80-d7ca7b23d6a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:34.104159 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855206f2-a0c6-4d74-ae80-d7ca7b23d6a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:34.104443 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855206f2-a0c6-4d74-ae80-d7ca7b23d6a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:34.104720 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855206f2-a0c6-4d74-ae80-d7ca7b23d6a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:34.113547 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855206f2-a0c6-4d74-ae80-d7ca7b23d6a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:34.114966 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855206f2-a0c6-4d74-ae80-d7ca7b23d6a1 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:34.124219 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1394/2785] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:25:34.389792 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf6e20-7446-4529-9a5c-e2c45e34da8f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:34.391263 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-edaf6e20-7446-4529-9a5c-e2c45e34da8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:34.405480 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf6e20-7446-4529-9a5c-e2c45e34da8f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:34.410111 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5ffc5-2d97-4d35-8df8-511e07e40a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:34.410967 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5ffc5-2d97-4d35-8df8-511e07e40a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:34.411253 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5ffc5-2d97-4d35-8df8-511e07e40a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:34.411529 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5ffc5-2d97-4d35-8df8-511e07e40a50 admin admin] PATH_INFO: `/v3/projects/eb0f79114b594ef5928b9be51715163e` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:34.412734 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f5ffc5-2d97-4d35-8df8-511e07e40a50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb0f79114b594ef5928b9be51715163e)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:34.414645 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f5ffc5-2d97-4d35-8df8-511e07e40a50 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:34.462343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e7f5ffc5-2d97-4d35-8df8-511e07e40a50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb0f79114b594ef5928b9be51715163e', 'request_id': 'req-e7f5ffc5-2d97-4d35-8df8-511e07e40a50'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:34.467905 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e7f5ffc5-2d97-4d35-8df8-511e07e40a50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb0f79114b594ef5928b9be51715163e', 'request_id': 'req-e7f5ffc5-2d97-4d35-8df8-511e07e40a50'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:34.476380 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e7f5ffc5-2d97-4d35-8df8-511e07e40a50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb0f79114b594ef5928b9be51715163e', 'request_id': 'req-e7f5ffc5-2d97-4d35-8df8-511e07e40a50'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:34.487239 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e7f5ffc5-2d97-4d35-8df8-511e07e40a50 admin admin] The token cache is being invalidate because project eb0f79114b594ef5928b9be51715163e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:25:34.487564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e7f5ffc5-2d97-4d35-8df8-511e07e40a50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:34.488223 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-e7f5ffc5-2d97-4d35-8df8-511e07e40a50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:34.490335 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1392/2786] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:25:34 2022] DELETE /identity/v3/projects/eb0f79114b594ef5928b9be51715163e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:34.494327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855206f2-a0c6-4d74-ae80-d7ca7b23d6a1 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:34.503896 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855206f2-a0c6-4d74-ae80-d7ca7b23d6a1 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:34.521289 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:34.560751 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4b730-ac82-4be3-8395-1ab05959e716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:34.561749 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4b730-ac82-4be3-8395-1ab05959e716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:34.562040 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4b730-ac82-4be3-8395-1ab05959e716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:34.562319 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4b730-ac82-4be3-8395-1ab05959e716 admin admin] PATH_INFO: `/v3/users/0efa165f910d4fa1ad72e7f01b2317d2` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:34.563511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4b730-ac82-4be3-8395-1ab05959e716 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0efa165f910d4fa1ad72e7f01b2317d2)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:34.565483 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4b730-ac82-4be3-8395-1ab05959e716 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:34.612973 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-36e4b730-ac82-4be3-8395-1ab05959e716 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0efa165f910d4fa1ad72e7f01b2317d2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:34.620256 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-36e4b730-ac82-4be3-8395-1ab05959e716 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0efa165f910d4fa1ad72e7f01b2317d2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:34.627243 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-36e4b730-ac82-4be3-8395-1ab05959e716 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0efa165f910d4fa1ad72e7f01b2317d2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:34.627903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-36e4b730-ac82-4be3-8395-1ab05959e716 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0efa165f910d4fa1ad72e7f01b2317d2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:34.639642 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-36e4b730-ac82-4be3-8395-1ab05959e716 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0efa165f910d4fa1ad72e7f01b2317d2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:34.647315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1395/2787] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:25:34 2022] DELETE /identity/v3/users/0efa165f910d4fa1ad72e7f01b2317d2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:34.912826 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5ffc5-2d97-4d35-8df8-511e07e40a50 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:34.914399 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-e7f5ffc5-2d97-4d35-8df8-511e07e40a50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:34.928362 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5ffc5-2d97-4d35-8df8-511e07e40a50 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:34.990874 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1a877-1196-4d9c-8174-dbfe2ac3622d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:34.991730 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1a877-1196-4d9c-8174-dbfe2ac3622d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:34.992050 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1a877-1196-4d9c-8174-dbfe2ac3622d admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:34.992333 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1a877-1196-4d9c-8174-dbfe2ac3622d admin admin] PATH_INFO: `/v3/projects/b4df57389fbe4f4382598e76184f410c` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:34.993552 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1a877-1196-4d9c-8174-dbfe2ac3622d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4df57389fbe4f4382598e76184f410c)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:34.995396 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1a877-1196-4d9c-8174-dbfe2ac3622d admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:35.025165 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:35.043176 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-25c1a877-1196-4d9c-8174-dbfe2ac3622d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4df57389fbe4f4382598e76184f410c', 'request_id': 'req-25c1a877-1196-4d9c-8174-dbfe2ac3622d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:35.048694 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-25c1a877-1196-4d9c-8174-dbfe2ac3622d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4df57389fbe4f4382598e76184f410c', 'request_id': 'req-25c1a877-1196-4d9c-8174-dbfe2ac3622d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:35.055925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-25c1a877-1196-4d9c-8174-dbfe2ac3622d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4df57389fbe4f4382598e76184f410c', 'request_id': 'req-25c1a877-1196-4d9c-8174-dbfe2ac3622d'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:35.067824 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-25c1a877-1196-4d9c-8174-dbfe2ac3622d admin admin] The token cache is being invalidate because project b4df57389fbe4f4382598e76184f410c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:25:35.068412 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-25c1a877-1196-4d9c-8174-dbfe2ac3622d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:35.069301 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-25c1a877-1196-4d9c-8174-dbfe2ac3622d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:35.071273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1393/2788] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:25:34 2022] DELETE /identity/v3/projects/b4df57389fbe4f4382598e76184f410c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:35.076225 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecda740-4280-4b59-b104-22d5561014ad None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:35.076703 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecda740-4280-4b59-b104-22d5561014ad None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:35.076811 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecda740-4280-4b59-b104-22d5561014ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:35.094635 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7ecda740-4280-4b59-b104-22d5561014ad None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:35.095847 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7ecda740-4280-4b59-b104-22d5561014ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:35.153394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1396/2789] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:25:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:35.161290 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1ff43-cf70-41c7-a483-0d0f85b4d2c8 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:35.161583 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1ff43-cf70-41c7-a483-0d0f85b4d2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:35.161870 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1ff43-cf70-41c7-a483-0d0f85b4d2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:35.179854 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-b1b1ff43-cf70-41c7-a483-0d0f85b4d2c8 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:35.181089 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-b1b1ff43-cf70-41c7-a483-0d0f85b4d2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:35.188992 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1394/2790] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:25:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:35.195089 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecda740-4280-4b59-b104-22d5561014ad None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:35.205190 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecda740-4280-4b59-b104-22d5561014ad None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:35.210710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e4ada-be3e-4540-8450-6cdea5e34963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:35.211357 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e4ada-be3e-4540-8450-6cdea5e34963 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:35.211649 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e4ada-be3e-4540-8450-6cdea5e34963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:35.211934 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e4ada-be3e-4540-8450-6cdea5e34963 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:35.212688 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8e4ada-be3e-4540-8450-6cdea5e34963 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:25:35.213004 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8e4ada-be3e-4540-8450-6cdea5e34963 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:35.214606 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8e4ada-be3e-4540-8450-6cdea5e34963 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:35.226068 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1397/2791] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:25:35.230191 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1ff43-cf70-41c7-a483-0d0f85b4d2c8 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:35.239252 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1ff43-cf70-41c7-a483-0d0f85b4d2c8 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:35.243970 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001bde8-a4bc-4685-9cb9-e914654299c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:35.244572 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001bde8-a4bc-4685-9cb9-e914654299c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:35.244866 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001bde8-a4bc-4685-9cb9-e914654299c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:35.245150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001bde8-a4bc-4685-9cb9-e914654299c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:35.245850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001bde8-a4bc-4685-9cb9-e914654299c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:35.247791 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001bde8-a4bc-4685-9cb9-e914654299c2 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:35.274102 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1395/2792] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:25:35 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:25:35.277417 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e4ada-be3e-4540-8450-6cdea5e34963 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:35.287183 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e4ada-be3e-4540-8450-6cdea5e34963 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:35.314091 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:35.347326 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd89d7d-f2c0-44c8-8e42-b85269d91a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:35.348033 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd89d7d-f2c0-44c8-8e42-b85269d91a47 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:35.348327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd89d7d-f2c0-44c8-8e42-b85269d91a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:35.348641 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd89d7d-f2c0-44c8-8e42-b85269d91a47 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:35.349332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd89d7d-f2c0-44c8-8e42-b85269d91a47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:35.351258 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd89d7d-f2c0-44c8-8e42-b85269d91a47 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:35.371070 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1398/2793] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:25:35 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:25:35.375103 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001bde8-a4bc-4685-9cb9-e914654299c2 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:35.385867 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001bde8-a4bc-4685-9cb9-e914654299c2 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:35.390462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4ed37-cfd5-4f3f-ad05-cfddcb91ce17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:35.391122 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4ed37-cfd5-4f3f-ad05-cfddcb91ce17 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:35.391412 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4ed37-cfd5-4f3f-ad05-cfddcb91ce17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:35.391695 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4ed37-cfd5-4f3f-ad05-cfddcb91ce17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:35.392465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e4ed37-cfd5-4f3f-ad05-cfddcb91ce17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:35.394063 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e4ed37-cfd5-4f3f-ad05-cfddcb91ce17 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:35.405038 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1396/2794] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:25:35.408849 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd89d7d-f2c0-44c8-8e42-b85269d91a47 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:35.419592 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd89d7d-f2c0-44c8-8e42-b85269d91a47 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:35.424256 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d94dd-e033-49f6-b6ec-66043a492ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:35.424911 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d94dd-e033-49f6-b6ec-66043a492ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:35.425204 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d94dd-e033-49f6-b6ec-66043a492ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:35.425488 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d94dd-e033-49f6-b6ec-66043a492ae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:35.426244 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d94dd-e033-49f6-b6ec-66043a492ae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:35.427897 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d94dd-e033-49f6-b6ec-66043a492ae5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:35.437769 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1399/2795] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:25:35.441738 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4ed37-cfd5-4f3f-ad05-cfddcb91ce17 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:35.451794 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4ed37-cfd5-4f3f-ad05-cfddcb91ce17 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:35.456322 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769cec59-544a-4f58-9899-2efa04278705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:35.457248 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769cec59-544a-4f58-9899-2efa04278705 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:35.457537 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769cec59-544a-4f58-9899-2efa04278705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:35.457818 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769cec59-544a-4f58-9899-2efa04278705 admin admin] PATH_INFO: `/v3/projects/618b2ac94ae4417296178018421c054c/users/defa86adf68d4fe393fa5fbe1190332b/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:35.472723 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769cec59-544a-4f58-9899-2efa04278705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=618b2ac94ae4417296178018421c054c, user_id=defa86adf68d4fe393fa5fbe1190332b, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:35.475971 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769cec59-544a-4f58-9899-2efa04278705 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:35.488846 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1397/2796] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:25:35 2022] PUT /identity/v3/projects/618b2ac94ae4417296178018421c054c/users/defa86adf68d4fe393fa5fbe1190332b/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:35.547723 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf4f81-cd2a-4d41-b639-85236a42c42e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:35.547937 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf4f81-cd2a-4d41-b639-85236a42c42e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:35.548233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf4f81-cd2a-4d41-b639-85236a42c42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:35.574397 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-56bf4f81-cd2a-4d41-b639-85236a42c42e None None] MFA Rules not processed for user `defa86adf68d4fe393fa5fbe1190332b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:35.575611 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-56bf4f81-cd2a-4d41-b639-85236a42c42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:35.638451 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1400/2797] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:25:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:35.642911 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769cec59-544a-4f58-9899-2efa04278705 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:35.707834 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769cec59-544a-4f58-9899-2efa04278705 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:35.713172 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6f9c1-7c9b-400c-a42d-befefe9fa907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:35.713835 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6f9c1-7c9b-400c-a42d-befefe9fa907 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:35.714127 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6f9c1-7c9b-400c-a42d-befefe9fa907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:35.714411 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6f9c1-7c9b-400c-a42d-befefe9fa907 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:35.715105 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6f9c1-7c9b-400c-a42d-befefe9fa907 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:35.717064 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6f9c1-7c9b-400c-a42d-befefe9fa907 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:35.741626 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1398/2798] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:25:35 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:25:35.746008 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf4f81-cd2a-4d41-b639-85236a42c42e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:35.757190 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf4f81-cd2a-4d41-b639-85236a42c42e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:35.814294 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c79a322-1de8-45c7-b9f0-3f9f8497ba7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:35.814898 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79a322-1de8-45c7-b9f0-3f9f8497ba7b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:35.815191 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79a322-1de8-45c7-b9f0-3f9f8497ba7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:35.815472 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79a322-1de8-45c7-b9f0-3f9f8497ba7b admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:35.816216 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c79a322-1de8-45c7-b9f0-3f9f8497ba7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:35.818666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c79a322-1de8-45c7-b9f0-3f9f8497ba7b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:35.839556 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1401/2799] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:25:35 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:25:35.843512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6f9c1-7c9b-400c-a42d-befefe9fa907 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:35.854007 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6f9c1-7c9b-400c-a42d-befefe9fa907 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:35.858792 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764716b4-bf06-419a-8289-e9e0bec0a4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:35.859452 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764716b4-bf06-419a-8289-e9e0bec0a4f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:35.859772 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764716b4-bf06-419a-8289-e9e0bec0a4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:35.860092 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764716b4-bf06-419a-8289-e9e0bec0a4f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:35.860833 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764716b4-bf06-419a-8289-e9e0bec0a4f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:35.862429 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764716b4-bf06-419a-8289-e9e0bec0a4f7 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:35.872986 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1399/2800] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:25:35.877096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c79a322-1de8-45c7-b9f0-3f9f8497ba7b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:35.887534 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c79a322-1de8-45c7-b9f0-3f9f8497ba7b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:35.892230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989dfdce-5e44-4b2d-aea0-bf860d154c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:35.893106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989dfdce-5e44-4b2d-aea0-bf860d154c96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:35.893396 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989dfdce-5e44-4b2d-aea0-bf860d154c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:35.893682 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989dfdce-5e44-4b2d-aea0-bf860d154c96 admin admin] PATH_INFO: `/v3/projects/08a6925f7d384eb1b8b181b3a5f6990d/users/2494c828a3774b939d15fe7192d5bdc6/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:35.908810 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989dfdce-5e44-4b2d-aea0-bf860d154c96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08a6925f7d384eb1b8b181b3a5f6990d, user_id=2494c828a3774b939d15fe7192d5bdc6, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:35.911478 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989dfdce-5e44-4b2d-aea0-bf860d154c96 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:35.924604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1402/2801] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:25:35 2022] PUT /identity/v3/projects/08a6925f7d384eb1b8b181b3a5f6990d/users/2494c828a3774b939d15fe7192d5bdc6/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:25:35.928627 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764716b4-bf06-419a-8289-e9e0bec0a4f7 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:35.938600 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764716b4-bf06-419a-8289-e9e0bec0a4f7 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:35.960389 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:35.997688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95195c-eb57-4473-a628-23ab4c1a8d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:35.998367 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95195c-eb57-4473-a628-23ab4c1a8d91 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:35.998641 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95195c-eb57-4473-a628-23ab4c1a8d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:35.998958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95195c-eb57-4473-a628-23ab4c1a8d91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:35.999689 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf95195c-eb57-4473-a628-23ab4c1a8d91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:36.001335 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf95195c-eb57-4473-a628-23ab4c1a8d91 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:36.011823 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1400/2802] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:25:36.015929 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989dfdce-5e44-4b2d-aea0-bf860d154c96 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:36.021969 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:36.028141 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989dfdce-5e44-4b2d-aea0-bf860d154c96 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:36.032922 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9848d3b-1630-45f5-96a8-d6297f7254c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:36.033793 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9848d3b-1630-45f5-96a8-d6297f7254c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:36.034080 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9848d3b-1630-45f5-96a8-d6297f7254c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:36.034358 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9848d3b-1630-45f5-96a8-d6297f7254c7 admin admin] PATH_INFO: `/v3/projects/08a6925f7d384eb1b8b181b3a5f6990d/users/2494c828a3774b939d15fe7192d5bdc6/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:36.036350 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9848d3b-1630-45f5-96a8-d6297f7254c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08a6925f7d384eb1b8b181b3a5f6990d, user_id=2494c828a3774b939d15fe7192d5bdc6, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:36.038960 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9848d3b-1630-45f5-96a8-d6297f7254c7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:36.048870 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1403/2803] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:25:36 2022] PUT /identity/v3/projects/08a6925f7d384eb1b8b181b3a5f6990d/users/2494c828a3774b939d15fe7192d5bdc6/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:25:36.059790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf795efa-7d99-455d-b7f4-f595c0df2cf9 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:36.060113 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf795efa-7d99-455d-b7f4-f595c0df2cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:36.060395 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf795efa-7d99-455d-b7f4-f595c0df2cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:36.086617 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-bf795efa-7d99-455d-b7f4-f595c0df2cf9 None None] MFA Rules not processed for user `2494c828a3774b939d15fe7192d5bdc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:36.087736 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bf795efa-7d99-455d-b7f4-f595c0df2cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:36.154595 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1401/2804] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:25:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:36.165018 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9848d3b-1630-45f5-96a8-d6297f7254c7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:36.166714 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b9848d3b-1630-45f5-96a8-d6297f7254c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:36.223620 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9848d3b-1630-45f5-96a8-d6297f7254c7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:36.227602 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aaa190-acce-469f-8e42-555555255bab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:36.228316 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aaa190-acce-469f-8e42-555555255bab service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:36.228637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aaa190-acce-469f-8e42-555555255bab service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:36.228947 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aaa190-acce-469f-8e42-555555255bab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:36.284829 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aaa190-acce-469f-8e42-555555255bab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:36.286275 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aaa190-acce-469f-8e42-555555255bab service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:36.295363 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1404/2805] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:25:41.915509 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf795efa-7d99-455d-b7f4-f595c0df2cf9 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:41.916994 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-bf795efa-7d99-455d-b7f4-f595c0df2cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:41.929951 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf795efa-7d99-455d-b7f4-f595c0df2cf9 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:41.933539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ddf458-44e3-4964-99cd-771de8651756 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:41.934176 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddf458-44e3-4964-99cd-771de8651756 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:41.934461 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddf458-44e3-4964-99cd-771de8651756 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:41.934738 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddf458-44e3-4964-99cd-771de8651756 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:41.943806 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ddf458-44e3-4964-99cd-771de8651756 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:41.945560 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ddf458-44e3-4964-99cd-771de8651756 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:41.955950 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1402/2806] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:25:46.291286 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:51.951446 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=58 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:10.788461 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aaa190-acce-469f-8e42-555555255bab service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:10.850522 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aaa190-acce-469f-8e42-555555255bab service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:10.856048 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd1a72-9287-4b2c-abe0-754a97467638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:10.856914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd1a72-9287-4b2c-abe0-754a97467638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:10.857201 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd1a72-9287-4b2c-abe0-754a97467638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:10.857484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd1a72-9287-4b2c-abe0-754a97467638 admin admin] PATH_INFO: `/v3/users/defa86adf68d4fe393fa5fbe1190332b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:10.858668 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd1a72-9287-4b2c-abe0-754a97467638 admin admin] RBAC: Authorizing `identity:delete_user(user_id=defa86adf68d4fe393fa5fbe1190332b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:10.860658 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd1a72-9287-4b2c-abe0-754a97467638 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:10.907596 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6efd1a72-9287-4b2c-abe0-754a97467638 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'defa86adf68d4fe393fa5fbe1190332b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:10.914329 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6efd1a72-9287-4b2c-abe0-754a97467638 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'defa86adf68d4fe393fa5fbe1190332b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:10.921102 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6efd1a72-9287-4b2c-abe0-754a97467638 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'defa86adf68d4fe393fa5fbe1190332b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:10.921790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6efd1a72-9287-4b2c-abe0-754a97467638 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'defa86adf68d4fe393fa5fbe1190332b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:10.935490 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6efd1a72-9287-4b2c-abe0-754a97467638 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'defa86adf68d4fe393fa5fbe1190332b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:10.941671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1405/2807] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:26:10 2022] DELETE /identity/v3/users/defa86adf68d4fe393fa5fbe1190332b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:10.946865 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcffe52-ff10-4267-aa90-38e95be34cfe None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:10.947331 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcffe52-ff10-4267-aa90-38e95be34cfe None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:10.947598 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcffe52-ff10-4267-aa90-38e95be34cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:11.090038 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-2dcffe52-ff10-4267-aa90-38e95be34cfe None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:11.091166 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2dcffe52-ff10-4267-aa90-38e95be34cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:11.149418 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1403/2808] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:11.159059 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd1a72-9287-4b2c-abe0-754a97467638 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:11.160512 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6efd1a72-9287-4b2c-abe0-754a97467638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:11.216051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd1a72-9287-4b2c-abe0-754a97467638 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:11.219574 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102e620-17b1-48d3-bbb7-93efb582956b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:11.220253 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102e620-17b1-48d3-bbb7-93efb582956b service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:11.220543 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102e620-17b1-48d3-bbb7-93efb582956b service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:11.220827 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102e620-17b1-48d3-bbb7-93efb582956b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:11.229727 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102e620-17b1-48d3-bbb7-93efb582956b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:11.231151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102e620-17b1-48d3-bbb7-93efb582956b service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:11.240471 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1406/2809] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:26:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:26:11.521405 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcffe52-ff10-4267-aa90-38e95be34cfe None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:11.522848 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2dcffe52-ff10-4267-aa90-38e95be34cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:11.536713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcffe52-ff10-4267-aa90-38e95be34cfe None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:11.541336 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9459a1f-0e4e-4abd-810e-c38b3730b449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:11.542194 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9459a1f-0e4e-4abd-810e-c38b3730b449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:11.542482 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9459a1f-0e4e-4abd-810e-c38b3730b449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:11.543312 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9459a1f-0e4e-4abd-810e-c38b3730b449 admin admin] PATH_INFO: `/v3/projects/618b2ac94ae4417296178018421c054c` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:11.544698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9459a1f-0e4e-4abd-810e-c38b3730b449 admin admin] RBAC: Authorizing `identity:delete_project(project_id=618b2ac94ae4417296178018421c054c)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:11.546599 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9459a1f-0e4e-4abd-810e-c38b3730b449 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:11.594577 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f9459a1f-0e4e-4abd-810e-c38b3730b449 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '618b2ac94ae4417296178018421c054c', 'request_id': 'req-f9459a1f-0e4e-4abd-810e-c38b3730b449'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:11.600067 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f9459a1f-0e4e-4abd-810e-c38b3730b449 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '618b2ac94ae4417296178018421c054c', 'request_id': 'req-f9459a1f-0e4e-4abd-810e-c38b3730b449'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:11.607733 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f9459a1f-0e4e-4abd-810e-c38b3730b449 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '618b2ac94ae4417296178018421c054c', 'request_id': 'req-f9459a1f-0e4e-4abd-810e-c38b3730b449'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:11.619508 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f9459a1f-0e4e-4abd-810e-c38b3730b449 admin admin] The token cache is being invalidate because project 618b2ac94ae4417296178018421c054c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:26:11.619798 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f9459a1f-0e4e-4abd-810e-c38b3730b449 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:11.620465 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-f9459a1f-0e4e-4abd-810e-c38b3730b449 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:11.622004 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1404/2810] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:26:11 2022] DELETE /identity/v3/projects/618b2ac94ae4417296178018421c054c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:11.626325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102e620-17b1-48d3-bbb7-93efb582956b service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:11.640552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102e620-17b1-48d3-bbb7-93efb582956b service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:11.658362 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:11.698016 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9aa0d3-bbb4-4486-8426-fb900b6473cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:11.698873 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9aa0d3-bbb4-4486-8426-fb900b6473cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:11.699156 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9aa0d3-bbb4-4486-8426-fb900b6473cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:11.699434 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9aa0d3-bbb4-4486-8426-fb900b6473cd admin admin] PATH_INFO: `/v3/users/2494c828a3774b939d15fe7192d5bdc6` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:11.700639 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9aa0d3-bbb4-4486-8426-fb900b6473cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2494c828a3774b939d15fe7192d5bdc6)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:11.702567 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9aa0d3-bbb4-4486-8426-fb900b6473cd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:11.748409 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7c9aa0d3-bbb4-4486-8426-fb900b6473cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2494c828a3774b939d15fe7192d5bdc6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:11.755610 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7c9aa0d3-bbb4-4486-8426-fb900b6473cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2494c828a3774b939d15fe7192d5bdc6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:11.762587 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7c9aa0d3-bbb4-4486-8426-fb900b6473cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2494c828a3774b939d15fe7192d5bdc6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:11.763220 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7c9aa0d3-bbb4-4486-8426-fb900b6473cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2494c828a3774b939d15fe7192d5bdc6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:11.774990 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7c9aa0d3-bbb4-4486-8426-fb900b6473cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2494c828a3774b939d15fe7192d5bdc6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:11.781228 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1407/2811] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:26:11 2022] DELETE /identity/v3/users/2494c828a3774b939d15fe7192d5bdc6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:12.057141 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9459a1f-0e4e-4abd-810e-c38b3730b449 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:12.058590 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-f9459a1f-0e4e-4abd-810e-c38b3730b449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:12.072233 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9459a1f-0e4e-4abd-810e-c38b3730b449 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:12.130790 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528180d0-6de9-4014-a6f4-af5f3bb91359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:12.131610 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528180d0-6de9-4014-a6f4-af5f3bb91359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:12.131898 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528180d0-6de9-4014-a6f4-af5f3bb91359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:12.132210 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528180d0-6de9-4014-a6f4-af5f3bb91359 admin admin] PATH_INFO: `/v3/projects/08a6925f7d384eb1b8b181b3a5f6990d` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:12.133408 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528180d0-6de9-4014-a6f4-af5f3bb91359 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08a6925f7d384eb1b8b181b3a5f6990d)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:12.135298 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528180d0-6de9-4014-a6f4-af5f3bb91359 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:12.166132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:12.183482 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-528180d0-6de9-4014-a6f4-af5f3bb91359 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08a6925f7d384eb1b8b181b3a5f6990d', 'request_id': 'req-528180d0-6de9-4014-a6f4-af5f3bb91359'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:12.188843 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-528180d0-6de9-4014-a6f4-af5f3bb91359 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08a6925f7d384eb1b8b181b3a5f6990d', 'request_id': 'req-528180d0-6de9-4014-a6f4-af5f3bb91359'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:12.196149 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-528180d0-6de9-4014-a6f4-af5f3bb91359 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08a6925f7d384eb1b8b181b3a5f6990d', 'request_id': 'req-528180d0-6de9-4014-a6f4-af5f3bb91359'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:12.206805 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-528180d0-6de9-4014-a6f4-af5f3bb91359 admin admin] The token cache is being invalidate because project 08a6925f7d384eb1b8b181b3a5f6990d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:26:12.207097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-528180d0-6de9-4014-a6f4-af5f3bb91359 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:12.207715 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-528180d0-6de9-4014-a6f4-af5f3bb91359 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:12.209243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1405/2812] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:26:12 2022] DELETE /identity/v3/projects/08a6925f7d384eb1b8b181b3a5f6990d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:12.214662 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298ce4b-4f71-4419-aa4d-a39374c44dc6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:12.215162 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298ce4b-4f71-4419-aa4d-a39374c44dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:12.215277 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298ce4b-4f71-4419-aa4d-a39374c44dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:12.233139 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e298ce4b-4f71-4419-aa4d-a39374c44dc6 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:12.234369 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e298ce4b-4f71-4419-aa4d-a39374c44dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:12.293257 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1408/2813] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:12.301102 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acea6de-dca3-46e4-9900-d4b6efc1f63b None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:12.301397 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acea6de-dca3-46e4-9900-d4b6efc1f63b None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:12.301681 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acea6de-dca3-46e4-9900-d4b6efc1f63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:12.319348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-9acea6de-dca3-46e4-9900-d4b6efc1f63b None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:12.320829 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-9acea6de-dca3-46e4-9900-d4b6efc1f63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:12.328616 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1406/2814] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:12.332693 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298ce4b-4f71-4419-aa4d-a39374c44dc6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:12.342755 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298ce4b-4f71-4419-aa4d-a39374c44dc6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:12.347365 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b79d2f-f87c-429a-b57c-01f10a60a141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:12.348042 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b79d2f-f87c-429a-b57c-01f10a60a141 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:12.348337 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b79d2f-f87c-429a-b57c-01f10a60a141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:12.348623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b79d2f-f87c-429a-b57c-01f10a60a141 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:12.349353 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b79d2f-f87c-429a-b57c-01f10a60a141 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:26:12.349669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b79d2f-f87c-429a-b57c-01f10a60a141 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:12.351819 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b79d2f-f87c-429a-b57c-01f10a60a141 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:12.363376 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1409/2815] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:26:12.367592 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acea6de-dca3-46e4-9900-d4b6efc1f63b None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:12.376632 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acea6de-dca3-46e4-9900-d4b6efc1f63b None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:12.381459 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13aa42-5248-4126-a43d-aee00159bfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:12.382077 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13aa42-5248-4126-a43d-aee00159bfe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:12.382353 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13aa42-5248-4126-a43d-aee00159bfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:12.383189 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13aa42-5248-4126-a43d-aee00159bfe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:12.383883 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13aa42-5248-4126-a43d-aee00159bfe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:12.385851 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13aa42-5248-4126-a43d-aee00159bfe3 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:12.410693 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1407/2816] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:26:12 2022] POST /identity/v3/projects => generated 401 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:26:12.414817 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b79d2f-f87c-429a-b57c-01f10a60a141 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:12.424717 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b79d2f-f87c-429a-b57c-01f10a60a141 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:12.450601 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:12.482916 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e96f9-427d-4c4f-b4e0-f852e37e9d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:12.483615 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e96f9-427d-4c4f-b4e0-f852e37e9d36 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:12.483906 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e96f9-427d-4c4f-b4e0-f852e37e9d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:12.484221 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e96f9-427d-4c4f-b4e0-f852e37e9d36 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:12.484922 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e96f9-427d-4c4f-b4e0-f852e37e9d36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:12.486838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e96f9-427d-4c4f-b4e0-f852e37e9d36 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:12.506214 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1410/2817] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:26:12 2022] POST /identity/v3/users => generated 301 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:26:12.510804 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13aa42-5248-4126-a43d-aee00159bfe3 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:12.520836 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13aa42-5248-4126-a43d-aee00159bfe3 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:12.526105 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c727f1-8818-42fb-82b6-8dd938a836fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:12.526775 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c727f1-8818-42fb-82b6-8dd938a836fd admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:12.527064 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c727f1-8818-42fb-82b6-8dd938a836fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:12.527347 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c727f1-8818-42fb-82b6-8dd938a836fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:12.528118 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c727f1-8818-42fb-82b6-8dd938a836fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:12.529800 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c727f1-8818-42fb-82b6-8dd938a836fd admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:12.540089 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1408/2818] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:26:12.543710 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e96f9-427d-4c4f-b4e0-f852e37e9d36 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:12.553559 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e96f9-427d-4c4f-b4e0-f852e37e9d36 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:12.558180 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf69861-83eb-41ef-ba4d-191907496d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:12.558828 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf69861-83eb-41ef-ba4d-191907496d96 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:12.559117 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf69861-83eb-41ef-ba4d-191907496d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:12.559396 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf69861-83eb-41ef-ba4d-191907496d96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:12.560158 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf69861-83eb-41ef-ba4d-191907496d96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:12.561734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf69861-83eb-41ef-ba4d-191907496d96 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:12.572168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1411/2819] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:26:12.575918 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c727f1-8818-42fb-82b6-8dd938a836fd admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:12.586313 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c727f1-8818-42fb-82b6-8dd938a836fd admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:12.591018 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbd30e-319d-4d1f-a8d1-1add585af24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:12.591891 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbd30e-319d-4d1f-a8d1-1add585af24f admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:12.592208 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbd30e-319d-4d1f-a8d1-1add585af24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:12.592547 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbd30e-319d-4d1f-a8d1-1add585af24f admin admin] PATH_INFO: `/v3/projects/a922c8a16b264594a0224c9d0aad0d40/users/3943d01918254a9f8d02db6149915723/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:12.606814 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bbd30e-319d-4d1f-a8d1-1add585af24f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a922c8a16b264594a0224c9d0aad0d40, user_id=3943d01918254a9f8d02db6149915723, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:12.609479 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bbd30e-319d-4d1f-a8d1-1add585af24f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:12.621912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1409/2820] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:26:12 2022] PUT /identity/v3/projects/a922c8a16b264594a0224c9d0aad0d40/users/3943d01918254a9f8d02db6149915723/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:12.632230 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a8204-c034-462e-8598-5d5dd8574a42 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:12.632964 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a8204-c034-462e-8598-5d5dd8574a42 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:12.633251 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a8204-c034-462e-8598-5d5dd8574a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:12.660720 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-122a8204-c034-462e-8598-5d5dd8574a42 None None] MFA Rules not processed for user `3943d01918254a9f8d02db6149915723`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:12.661937 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-122a8204-c034-462e-8598-5d5dd8574a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:12.725871 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1412/2821] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:26:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:12.730348 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbd30e-319d-4d1f-a8d1-1add585af24f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:12.791200 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbd30e-319d-4d1f-a8d1-1add585af24f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:12.796115 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c07c07-adb8-450a-a1b3-cefbb97c9042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:12.796782 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c07c07-adb8-450a-a1b3-cefbb97c9042 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:12.797165 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c07c07-adb8-450a-a1b3-cefbb97c9042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:12.797546 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c07c07-adb8-450a-a1b3-cefbb97c9042 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:12.798217 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c07c07-adb8-450a-a1b3-cefbb97c9042 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:12.800369 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c07c07-adb8-450a-a1b3-cefbb97c9042 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:12.826154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1410/2822] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:26:12 2022] POST /identity/v3/projects => generated 401 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:26:12.830186 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a8204-c034-462e-8598-5d5dd8574a42 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:12.840748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a8204-c034-462e-8598-5d5dd8574a42 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:12.896846 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a601a7a-f6e4-4e58-aeeb-9eb98975e28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:12.897446 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a601a7a-f6e4-4e58-aeeb-9eb98975e28d admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:12.897738 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a601a7a-f6e4-4e58-aeeb-9eb98975e28d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:12.898024 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a601a7a-f6e4-4e58-aeeb-9eb98975e28d admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:12.898722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a601a7a-f6e4-4e58-aeeb-9eb98975e28d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:12.900694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a601a7a-f6e4-4e58-aeeb-9eb98975e28d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:12.920669 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1413/2823] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:26:12 2022] POST /identity/v3/users => generated 307 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:26:12.924766 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c07c07-adb8-450a-a1b3-cefbb97c9042 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:12.935596 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c07c07-adb8-450a-a1b3-cefbb97c9042 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:12.940284 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1be21-901d-4f3e-9102-f2e91603d52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:12.940949 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1be21-901d-4f3e-9102-f2e91603d52b admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:12.941242 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1be21-901d-4f3e-9102-f2e91603d52b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:12.941526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1be21-901d-4f3e-9102-f2e91603d52b admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:12.942260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1be21-901d-4f3e-9102-f2e91603d52b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:12.943864 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1be21-901d-4f3e-9102-f2e91603d52b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:12.955514 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1411/2824] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:26:12.959707 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a601a7a-f6e4-4e58-aeeb-9eb98975e28d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:12.969965 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a601a7a-f6e4-4e58-aeeb-9eb98975e28d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:12.974666 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7d4bc-a2ab-4967-938c-0a33fe7f92f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:12.975541 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7d4bc-a2ab-4967-938c-0a33fe7f92f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:12.975830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7d4bc-a2ab-4967-938c-0a33fe7f92f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:12.976251 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7d4bc-a2ab-4967-938c-0a33fe7f92f7 admin admin] PATH_INFO: `/v3/projects/ea6ef73249d84f88963cf34916325f3a/users/98bbe85b781b41879598458fe52c8ed0/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:12.991163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7d4bc-a2ab-4967-938c-0a33fe7f92f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea6ef73249d84f88963cf34916325f3a, user_id=98bbe85b781b41879598458fe52c8ed0, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:12.994471 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7d4bc-a2ab-4967-938c-0a33fe7f92f7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:13.007047 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1414/2825] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:26:12 2022] PUT /identity/v3/projects/ea6ef73249d84f88963cf34916325f3a/users/98bbe85b781b41879598458fe52c8ed0/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:13.011512 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1be21-901d-4f3e-9102-f2e91603d52b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:13.021396 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1be21-901d-4f3e-9102-f2e91603d52b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:13.042479 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:13.080871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ca05e-c3c6-41ee-b70d-b9ba4d9b11ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:13.081564 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ca05e-c3c6-41ee-b70d-b9ba4d9b11ad admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:13.081811 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ca05e-c3c6-41ee-b70d-b9ba4d9b11ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:13.082088 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ca05e-c3c6-41ee-b70d-b9ba4d9b11ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:13.082810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ca05e-c3c6-41ee-b70d-b9ba4d9b11ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:13.084443 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ca05e-c3c6-41ee-b70d-b9ba4d9b11ad admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:13.093951 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1412/2826] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:26:13.098141 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7d4bc-a2ab-4967-938c-0a33fe7f92f7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:13.104273 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:13.110359 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7d4bc-a2ab-4967-938c-0a33fe7f92f7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:13.115173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172eef35-c6ab-4e72-8e45-8155c0341f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:13.116078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172eef35-c6ab-4e72-8e45-8155c0341f38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:13.116362 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172eef35-c6ab-4e72-8e45-8155c0341f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:13.116637 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172eef35-c6ab-4e72-8e45-8155c0341f38 admin admin] PATH_INFO: `/v3/projects/ea6ef73249d84f88963cf34916325f3a/users/98bbe85b781b41879598458fe52c8ed0/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:13.119188 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172eef35-c6ab-4e72-8e45-8155c0341f38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea6ef73249d84f88963cf34916325f3a, user_id=98bbe85b781b41879598458fe52c8ed0, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:13.121838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172eef35-c6ab-4e72-8e45-8155c0341f38 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:13.131827 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1415/2827] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:26:13 2022] PUT /identity/v3/projects/ea6ef73249d84f88963cf34916325f3a/users/98bbe85b781b41879598458fe52c8ed0/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:13.142631 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0d5cb-5756-41cf-90ca-4e86273fd7ff None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:13.142924 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0d5cb-5756-41cf-90ca-4e86273fd7ff None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:13.143206 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0d5cb-5756-41cf-90ca-4e86273fd7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:13.169001 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-31e0d5cb-5756-41cf-90ca-4e86273fd7ff None None] MFA Rules not processed for user `98bbe85b781b41879598458fe52c8ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:13.170174 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-31e0d5cb-5756-41cf-90ca-4e86273fd7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:13.241429 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1413/2828] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:26:13 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:13.252501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172eef35-c6ab-4e72-8e45-8155c0341f38 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:13.254255 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-172eef35-c6ab-4e72-8e45-8155c0341f38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:13.314122 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172eef35-c6ab-4e72-8e45-8155c0341f38 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:13.317864 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ab4fa-ac36-4fb1-b941-48e992685485 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:13.318514 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ab4fa-ac36-4fb1-b941-48e992685485 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:13.318806 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ab4fa-ac36-4fb1-b941-48e992685485 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:13.319091 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ab4fa-ac36-4fb1-b941-48e992685485 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:13.328332 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ab4fa-ac36-4fb1-b941-48e992685485 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:13.329890 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ab4fa-ac36-4fb1-b941-48e992685485 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:13.339814 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1416/2829] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:26:13 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:26:13.411884 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0d5cb-5756-41cf-90ca-4e86273fd7ff None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:13.421982 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0d5cb-5756-41cf-90ca-4e86273fd7ff None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:13.425459 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15cd1d-9797-43b5-82a1-15af4a71c5f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:13.426091 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15cd1d-9797-43b5-82a1-15af4a71c5f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:13.426373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15cd1d-9797-43b5-82a1-15af4a71c5f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:13.426651 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15cd1d-9797-43b5-82a1-15af4a71c5f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:13.481911 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15cd1d-9797-43b5-82a1-15af4a71c5f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:13.483344 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15cd1d-9797-43b5-82a1-15af4a71c5f2 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:13.492938 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1414/2830] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:26:13 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:26:17.510453 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ab4fa-ac36-4fb1-b941-48e992685485 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:17.571351 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ab4fa-ac36-4fb1-b941-48e992685485 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:17.575798 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703456a-b8b6-4822-8eba-52570d33997b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:17.576788 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703456a-b8b6-4822-8eba-52570d33997b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:17.577074 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703456a-b8b6-4822-8eba-52570d33997b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:17.577355 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703456a-b8b6-4822-8eba-52570d33997b admin admin] PATH_INFO: `/v3/users/3943d01918254a9f8d02db6149915723` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:17.578524 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703456a-b8b6-4822-8eba-52570d33997b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3943d01918254a9f8d02db6149915723)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:17.580476 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703456a-b8b6-4822-8eba-52570d33997b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:17.626137 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c703456a-b8b6-4822-8eba-52570d33997b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3943d01918254a9f8d02db6149915723'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:17.633256 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c703456a-b8b6-4822-8eba-52570d33997b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3943d01918254a9f8d02db6149915723'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:17.640029 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c703456a-b8b6-4822-8eba-52570d33997b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3943d01918254a9f8d02db6149915723'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:17.640657 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c703456a-b8b6-4822-8eba-52570d33997b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3943d01918254a9f8d02db6149915723'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:17.652186 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c703456a-b8b6-4822-8eba-52570d33997b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3943d01918254a9f8d02db6149915723'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:17.657954 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1417/2831] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:26:17 2022] DELETE /identity/v3/users/3943d01918254a9f8d02db6149915723 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:17.662397 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c95c4a-2904-4f99-b9e2-a0aaab524efe None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:17.662870 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c95c4a-2904-4f99-b9e2-a0aaab524efe None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:17.662984 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c95c4a-2904-4f99-b9e2-a0aaab524efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:17.680653 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-36c95c4a-2904-4f99-b9e2-a0aaab524efe None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:17.681858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-36c95c4a-2904-4f99-b9e2-a0aaab524efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:17.721017 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:17.740069 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1415/2832] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:17.750333 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703456a-b8b6-4822-8eba-52570d33997b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:17.751728 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c703456a-b8b6-4822-8eba-52570d33997b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:17.765130 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:17.812315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703456a-b8b6-4822-8eba-52570d33997b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:17.815955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796bb0a-6a7a-4e3f-b948-5088a608a425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:17.816613 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796bb0a-6a7a-4e3f-b948-5088a608a425 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:17.816904 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796bb0a-6a7a-4e3f-b948-5088a608a425 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:17.817189 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796bb0a-6a7a-4e3f-b948-5088a608a425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:17.826648 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d796bb0a-6a7a-4e3f-b948-5088a608a425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:17.828124 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d796bb0a-6a7a-4e3f-b948-5088a608a425 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:17.837560 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1418/2833] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:26:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:26:18.102472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c95c4a-2904-4f99-b9e2-a0aaab524efe None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:18.103934 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-36c95c4a-2904-4f99-b9e2-a0aaab524efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:18.118553 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c95c4a-2904-4f99-b9e2-a0aaab524efe None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:18.123502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837589b9-3760-4e95-b390-42c56ddf8959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:18.124602 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837589b9-3760-4e95-b390-42c56ddf8959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:18.124898 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837589b9-3760-4e95-b390-42c56ddf8959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:18.125181 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837589b9-3760-4e95-b390-42c56ddf8959 admin admin] PATH_INFO: `/v3/projects/a922c8a16b264594a0224c9d0aad0d40` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:18.126503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837589b9-3760-4e95-b390-42c56ddf8959 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a922c8a16b264594a0224c9d0aad0d40)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:18.128508 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837589b9-3760-4e95-b390-42c56ddf8959 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:18.174737 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-837589b9-3760-4e95-b390-42c56ddf8959 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a922c8a16b264594a0224c9d0aad0d40', 'request_id': 'req-837589b9-3760-4e95-b390-42c56ddf8959'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:18.180343 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-837589b9-3760-4e95-b390-42c56ddf8959 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a922c8a16b264594a0224c9d0aad0d40', 'request_id': 'req-837589b9-3760-4e95-b390-42c56ddf8959'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:18.187555 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-837589b9-3760-4e95-b390-42c56ddf8959 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a922c8a16b264594a0224c9d0aad0d40', 'request_id': 'req-837589b9-3760-4e95-b390-42c56ddf8959'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:18.199474 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-837589b9-3760-4e95-b390-42c56ddf8959 admin admin] The token cache is being invalidate because project a922c8a16b264594a0224c9d0aad0d40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:26:18.199763 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-837589b9-3760-4e95-b390-42c56ddf8959 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:18.200434 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-837589b9-3760-4e95-b390-42c56ddf8959 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:18.201954 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1416/2834] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:26:18 2022] DELETE /identity/v3/projects/a922c8a16b264594a0224c9d0aad0d40 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:18.205901 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796bb0a-6a7a-4e3f-b948-5088a608a425 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:18.215916 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796bb0a-6a7a-4e3f-b948-5088a608a425 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:18.270237 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f11e6-d5f8-4096-a591-75d9ef54db39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:18.271098 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f11e6-d5f8-4096-a591-75d9ef54db39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:18.271386 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f11e6-d5f8-4096-a591-75d9ef54db39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:18.271665 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f11e6-d5f8-4096-a591-75d9ef54db39 admin admin] PATH_INFO: `/v3/users/98bbe85b781b41879598458fe52c8ed0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:18.272876 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f11e6-d5f8-4096-a591-75d9ef54db39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98bbe85b781b41879598458fe52c8ed0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:18.275327 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2f11e6-d5f8-4096-a591-75d9ef54db39 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:18.321113 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7f2f11e6-d5f8-4096-a591-75d9ef54db39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98bbe85b781b41879598458fe52c8ed0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:18.327794 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7f2f11e6-d5f8-4096-a591-75d9ef54db39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98bbe85b781b41879598458fe52c8ed0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:18.334604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7f2f11e6-d5f8-4096-a591-75d9ef54db39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98bbe85b781b41879598458fe52c8ed0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:18.335235 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7f2f11e6-d5f8-4096-a591-75d9ef54db39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98bbe85b781b41879598458fe52c8ed0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:18.347549 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7f2f11e6-d5f8-4096-a591-75d9ef54db39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98bbe85b781b41879598458fe52c8ed0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:18.354097 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1419/2835] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:26:18 2022] DELETE /identity/v3/users/98bbe85b781b41879598458fe52c8ed0 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:18.616609 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837589b9-3760-4e95-b390-42c56ddf8959 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:18.618062 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-837589b9-3760-4e95-b390-42c56ddf8959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:18.631974 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837589b9-3760-4e95-b390-42c56ddf8959 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:18.688610 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c3ff2-ec17-4262-a8ce-bdcd26f081e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:18.689471 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c3ff2-ec17-4262-a8ce-bdcd26f081e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:18.689757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c3ff2-ec17-4262-a8ce-bdcd26f081e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:18.690037 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c3ff2-ec17-4262-a8ce-bdcd26f081e5 admin admin] PATH_INFO: `/v3/projects/ea6ef73249d84f88963cf34916325f3a` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:18.691222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c3ff2-ec17-4262-a8ce-bdcd26f081e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea6ef73249d84f88963cf34916325f3a)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:18.693164 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c3ff2-ec17-4262-a8ce-bdcd26f081e5 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:18.738256 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ee6c3ff2-ec17-4262-a8ce-bdcd26f081e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea6ef73249d84f88963cf34916325f3a', 'request_id': 'req-ee6c3ff2-ec17-4262-a8ce-bdcd26f081e5'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:18.744269 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ee6c3ff2-ec17-4262-a8ce-bdcd26f081e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea6ef73249d84f88963cf34916325f3a', 'request_id': 'req-ee6c3ff2-ec17-4262-a8ce-bdcd26f081e5'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:18.751162 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ee6c3ff2-ec17-4262-a8ce-bdcd26f081e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea6ef73249d84f88963cf34916325f3a', 'request_id': 'req-ee6c3ff2-ec17-4262-a8ce-bdcd26f081e5'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:18.758526 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:18.763561 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ee6c3ff2-ec17-4262-a8ce-bdcd26f081e5 admin admin] The token cache is being invalidate because project ea6ef73249d84f88963cf34916325f3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:26:18.763885 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ee6c3ff2-ec17-4262-a8ce-bdcd26f081e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:18.764789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-ee6c3ff2-ec17-4262-a8ce-bdcd26f081e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:18.766702 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1417/2836] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:26:18 2022] DELETE /identity/v3/projects/ea6ef73249d84f88963cf34916325f3a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:18.771845 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93274aaa-cc15-4503-91da-7427c076949c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:18.772346 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93274aaa-cc15-4503-91da-7427c076949c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:18.772443 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93274aaa-cc15-4503-91da-7427c076949c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:18.790043 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-93274aaa-cc15-4503-91da-7427c076949c None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:18.791292 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-93274aaa-cc15-4503-91da-7427c076949c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:18.823248 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:18.854770 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1420/2837] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:18.862827 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810286c5-ab1d-4d0f-bc0b-e3ae31f68435 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:18.863149 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810286c5-ab1d-4d0f-bc0b-e3ae31f68435 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:18.863430 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810286c5-ab1d-4d0f-bc0b-e3ae31f68435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:18.882046 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-810286c5-ab1d-4d0f-bc0b-e3ae31f68435 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:18.883934 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-810286c5-ab1d-4d0f-bc0b-e3ae31f68435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:18.891916 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1418/2838] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:18.896160 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93274aaa-cc15-4503-91da-7427c076949c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:18.905808 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93274aaa-cc15-4503-91da-7427c076949c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:18.910362 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaef9e4-f046-4e37-a123-98e3abeb45d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:18.911010 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaef9e4-f046-4e37-a123-98e3abeb45d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:18.911300 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaef9e4-f046-4e37-a123-98e3abeb45d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:18.911583 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaef9e4-f046-4e37-a123-98e3abeb45d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:18.912337 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaef9e4-f046-4e37-a123-98e3abeb45d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:26:18.912653 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaef9e4-f046-4e37-a123-98e3abeb45d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:18.914243 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaef9e4-f046-4e37-a123-98e3abeb45d0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:18.925583 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1421/2839] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:26:18.930244 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810286c5-ab1d-4d0f-bc0b-e3ae31f68435 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:18.939574 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810286c5-ab1d-4d0f-bc0b-e3ae31f68435 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:18.944262 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93fa988-b964-47de-bb72-bb2372761155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:18.945633 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fa988-b964-47de-bb72-bb2372761155 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:18.945928 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fa988-b964-47de-bb72-bb2372761155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:18.946205 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fa988-b964-47de-bb72-bb2372761155 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:18.946906 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93fa988-b964-47de-bb72-bb2372761155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:18.948819 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93fa988-b964-47de-bb72-bb2372761155 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:18.973731 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1419/2840] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:26:18 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:26:18.977908 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaef9e4-f046-4e37-a123-98e3abeb45d0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:18.988103 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaef9e4-f046-4e37-a123-98e3abeb45d0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:19.044011 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bb285-8365-4224-ab6b-bb70a56595a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:19.044618 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bb285-8365-4224-ab6b-bb70a56595a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:19.044908 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bb285-8365-4224-ab6b-bb70a56595a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:19.045189 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bb285-8365-4224-ab6b-bb70a56595a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:19.045882 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677bb285-8365-4224-ab6b-bb70a56595a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:19.047791 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677bb285-8365-4224-ab6b-bb70a56595a9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:19.067247 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1422/2841] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:26:18 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:26:19.071912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93fa988-b964-47de-bb72-bb2372761155 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:19.082156 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93fa988-b964-47de-bb72-bb2372761155 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:19.087020 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3a614-cf63-469c-99a1-32bb6031817b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:19.087677 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3a614-cf63-469c-99a1-32bb6031817b admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:19.087990 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3a614-cf63-469c-99a1-32bb6031817b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:19.088279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3a614-cf63-469c-99a1-32bb6031817b admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:19.089009 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c3a614-cf63-469c-99a1-32bb6031817b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:19.090599 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c3a614-cf63-469c-99a1-32bb6031817b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:19.101611 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1420/2842] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:26:19.105296 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bb285-8365-4224-ab6b-bb70a56595a9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:19.115022 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bb285-8365-4224-ab6b-bb70a56595a9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:19.120243 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93977ea8-03da-4c2e-9a64-b8042c5442dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:19.120886 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93977ea8-03da-4c2e-9a64-b8042c5442dd admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:19.121175 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93977ea8-03da-4c2e-9a64-b8042c5442dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:19.121456 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93977ea8-03da-4c2e-9a64-b8042c5442dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:19.122176 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93977ea8-03da-4c2e-9a64-b8042c5442dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:19.123752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93977ea8-03da-4c2e-9a64-b8042c5442dd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:19.134017 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1423/2843] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:26:19.138248 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3a614-cf63-469c-99a1-32bb6031817b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:19.148417 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3a614-cf63-469c-99a1-32bb6031817b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:19.153046 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b92f65-0d6a-4d0d-bdd4-d62f76f1fd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:19.153935 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b92f65-0d6a-4d0d-bdd4-d62f76f1fd6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:19.154229 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b92f65-0d6a-4d0d-bdd4-d62f76f1fd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:19.154507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b92f65-0d6a-4d0d-bdd4-d62f76f1fd6a admin admin] PATH_INFO: `/v3/projects/3ae9c262531e496286a1fcad0ce8e633/users/5bfbf8185ec34aacb47116b3866cc472/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:19.170373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b92f65-0d6a-4d0d-bdd4-d62f76f1fd6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ae9c262531e496286a1fcad0ce8e633, user_id=5bfbf8185ec34aacb47116b3866cc472, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:19.173129 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b92f65-0d6a-4d0d-bdd4-d62f76f1fd6a admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:19.187901 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1421/2844] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:26:19 2022] PUT /identity/v3/projects/3ae9c262531e496286a1fcad0ce8e633/users/5bfbf8185ec34aacb47116b3866cc472/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:19.193086 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faaf7a-391f-4cd1-ba5c-e537d8368982 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:19.193379 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faaf7a-391f-4cd1-ba5c-e537d8368982 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:19.193677 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faaf7a-391f-4cd1-ba5c-e537d8368982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:19.211467 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-13faaf7a-391f-4cd1-ba5c-e537d8368982 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:19.212698 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-13faaf7a-391f-4cd1-ba5c-e537d8368982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:19.271445 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1424/2845] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:19.280945 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b92f65-0d6a-4d0d-bdd4-d62f76f1fd6a admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:19.282611 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-28b92f65-0d6a-4d0d-bdd4-d62f76f1fd6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:19.338808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b92f65-0d6a-4d0d-bdd4-d62f76f1fd6a admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:19.342395 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f66327-0350-41aa-aa76-587d2a982b6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:19.343038 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f66327-0350-41aa-aa76-587d2a982b6e service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:19.343327 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f66327-0350-41aa-aa76-587d2a982b6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:19.343610 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f66327-0350-41aa-aa76-587d2a982b6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:19.353567 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f66327-0350-41aa-aa76-587d2a982b6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:19.354999 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f66327-0350-41aa-aa76-587d2a982b6e service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:19.364447 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1422/2846] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:26:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:26:19.965892 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e48a2-080b-4c0d-8b7b-68dd20369d73 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:19.966407 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e48a2-080b-4c0d-8b7b-68dd20369d73 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:19.966528 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e48a2-080b-4c0d-8b7b-68dd20369d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:19.992943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ca3e48a2-080b-4c0d-8b7b-68dd20369d73 None None] MFA Rules not processed for user `5bfbf8185ec34aacb47116b3866cc472`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:19.994319 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ca3e48a2-080b-4c0d-8b7b-68dd20369d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:20.023976 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:20.062887 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1425/2847] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:26:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:20.067427 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f66327-0350-41aa-aa76-587d2a982b6e service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:20.077710 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f66327-0350-41aa-aa76-587d2a982b6e service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:20.082279 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823597c-6c59-49d9-8ba8-c76056bfb153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:20.082927 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823597c-6c59-49d9-8ba8-c76056bfb153 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:20.083215 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823597c-6c59-49d9-8ba8-c76056bfb153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:20.083495 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823597c-6c59-49d9-8ba8-c76056bfb153 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:20.084237 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823597c-6c59-49d9-8ba8-c76056bfb153 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:20.086679 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823597c-6c59-49d9-8ba8-c76056bfb153 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:20.112958 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1423/2848] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:26:20 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:26:20.117096 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e48a2-080b-4c0d-8b7b-68dd20369d73 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:20.127674 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e48a2-080b-4c0d-8b7b-68dd20369d73 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:20.185743 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb177536-d52a-43fc-986c-c8a9d681c429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:20.186372 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb177536-d52a-43fc-986c-c8a9d681c429 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:20.186658 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb177536-d52a-43fc-986c-c8a9d681c429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:20.186996 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb177536-d52a-43fc-986c-c8a9d681c429 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:20.187650 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb177536-d52a-43fc-986c-c8a9d681c429 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:20.189657 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb177536-d52a-43fc-986c-c8a9d681c429 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:20.214859 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1426/2849] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:26:20 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:26:20.216047 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823597c-6c59-49d9-8ba8-c76056bfb153 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:20.226858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823597c-6c59-49d9-8ba8-c76056bfb153 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:20.231560 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90bb51-c825-4d03-9f22-5ccce8146f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:20.232273 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90bb51-c825-4d03-9f22-5ccce8146f35 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:20.232561 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90bb51-c825-4d03-9f22-5ccce8146f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:20.232839 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90bb51-c825-4d03-9f22-5ccce8146f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:20.233570 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90bb51-c825-4d03-9f22-5ccce8146f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:20.235173 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90bb51-c825-4d03-9f22-5ccce8146f35 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:20.247622 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1424/2850] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:26:20.251860 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb177536-d52a-43fc-986c-c8a9d681c429 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:20.262937 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb177536-d52a-43fc-986c-c8a9d681c429 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:20.267648 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca5eeb-983d-4818-a7a5-4069b7a6aaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:20.268573 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca5eeb-983d-4818-a7a5-4069b7a6aaa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:20.268860 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca5eeb-983d-4818-a7a5-4069b7a6aaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:20.269264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca5eeb-983d-4818-a7a5-4069b7a6aaa2 admin admin] PATH_INFO: `/v3/projects/88b56e01548c40dfabfe48ce189d7bc9/users/89b77e7d07894c6a9940732d035eeaaa/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:20.284233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca5eeb-983d-4818-a7a5-4069b7a6aaa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88b56e01548c40dfabfe48ce189d7bc9, user_id=89b77e7d07894c6a9940732d035eeaaa, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:20.286890 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca5eeb-983d-4818-a7a5-4069b7a6aaa2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:20.299617 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1427/2851] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:26:20 2022] PUT /identity/v3/projects/88b56e01548c40dfabfe48ce189d7bc9/users/89b77e7d07894c6a9940732d035eeaaa/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:20.303548 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90bb51-c825-4d03-9f22-5ccce8146f35 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:20.313952 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90bb51-c825-4d03-9f22-5ccce8146f35 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:20.368809 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716199b-d433-4cd4-9891-7e0722486d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:20.369503 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716199b-d433-4cd4-9891-7e0722486d25 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:20.369791 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716199b-d433-4cd4-9891-7e0722486d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:20.370071 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716199b-d433-4cd4-9891-7e0722486d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:20.370786 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716199b-d433-4cd4-9891-7e0722486d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:20.372405 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716199b-d433-4cd4-9891-7e0722486d25 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:20.377334 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:20.383108 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1425/2852] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:26:20.387238 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca5eeb-983d-4818-a7a5-4069b7a6aaa2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:20.397486 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca5eeb-983d-4818-a7a5-4069b7a6aaa2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:20.402195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b697d-8425-45ac-b578-8914635cadb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:20.403099 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b697d-8425-45ac-b578-8914635cadb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:20.403384 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b697d-8425-45ac-b578-8914635cadb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:20.403694 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b697d-8425-45ac-b578-8914635cadb8 admin admin] PATH_INFO: `/v3/projects/88b56e01548c40dfabfe48ce189d7bc9/users/89b77e7d07894c6a9940732d035eeaaa/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:20.405759 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b697d-8425-45ac-b578-8914635cadb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88b56e01548c40dfabfe48ce189d7bc9, user_id=89b77e7d07894c6a9940732d035eeaaa, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:20.408432 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b697d-8425-45ac-b578-8914635cadb8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:20.418412 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1428/2853] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:26:20 2022] PUT /identity/v3/projects/88b56e01548c40dfabfe48ce189d7bc9/users/89b77e7d07894c6a9940732d035eeaaa/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:21.007130 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18789721-8603-457f-b46d-5fe996ee72e5 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:21.007431 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18789721-8603-457f-b46d-5fe996ee72e5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:21.007713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18789721-8603-457f-b46d-5fe996ee72e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:21.034567 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-18789721-8603-457f-b46d-5fe996ee72e5 None None] MFA Rules not processed for user `89b77e7d07894c6a9940732d035eeaaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:21.035799 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-18789721-8603-457f-b46d-5fe996ee72e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:21.106288 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1426/2854] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:26:21 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:21.117673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b697d-8425-45ac-b578-8914635cadb8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:21.119454 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bc8b697d-8425-45ac-b578-8914635cadb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:21.181060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b697d-8425-45ac-b578-8914635cadb8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:21.185133 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5bdc8b-acea-4fe3-b09e-9c0f3cfc4337 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:21.185792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bdc8b-acea-4fe3-b09e-9c0f3cfc4337 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:21.186079 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bdc8b-acea-4fe3-b09e-9c0f3cfc4337 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:21.186357 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bdc8b-acea-4fe3-b09e-9c0f3cfc4337 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:21.220215 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:21.243810 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5bdc8b-acea-4fe3-b09e-9c0f3cfc4337 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:21.245321 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5bdc8b-acea-4fe3-b09e-9c0f3cfc4337 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:21.254890 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1429/2855] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:26:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:26:22.593935 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18789721-8603-457f-b46d-5fe996ee72e5 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:22.595319 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-18789721-8603-457f-b46d-5fe996ee72e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:22.607704 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18789721-8603-457f-b46d-5fe996ee72e5 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:22.611200 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f859f-e82c-4e78-b1c3-4219fc5a3728 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:22.611838 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f859f-e82c-4e78-b1c3-4219fc5a3728 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:22.612222 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f859f-e82c-4e78-b1c3-4219fc5a3728 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:22.612506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f859f-e82c-4e78-b1c3-4219fc5a3728 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:22.621337 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f859f-e82c-4e78-b1c3-4219fc5a3728 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:22.622757 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f859f-e82c-4e78-b1c3-4219fc5a3728 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:22.632151 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1427/2856] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:26:22 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:26:31.250233 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=11 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:32.627937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=40 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:35.440477 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5bdc8b-acea-4fe3-b09e-9c0f3cfc4337 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:35.502923 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5bdc8b-acea-4fe3-b09e-9c0f3cfc4337 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:35.508706 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cacd2fe-8e5e-4e12-9100-0473bf56a052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:35.509576 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacd2fe-8e5e-4e12-9100-0473bf56a052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:35.509867 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacd2fe-8e5e-4e12-9100-0473bf56a052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:35.510158 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacd2fe-8e5e-4e12-9100-0473bf56a052 admin admin] PATH_INFO: `/v3/users/5bfbf8185ec34aacb47116b3866cc472` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:35.511324 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cacd2fe-8e5e-4e12-9100-0473bf56a052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bfbf8185ec34aacb47116b3866cc472)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:35.513258 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cacd2fe-8e5e-4e12-9100-0473bf56a052 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:35.557763 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0cacd2fe-8e5e-4e12-9100-0473bf56a052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bfbf8185ec34aacb47116b3866cc472'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:35.565540 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0cacd2fe-8e5e-4e12-9100-0473bf56a052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bfbf8185ec34aacb47116b3866cc472'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:35.573120 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0cacd2fe-8e5e-4e12-9100-0473bf56a052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bfbf8185ec34aacb47116b3866cc472'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:35.573759 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0cacd2fe-8e5e-4e12-9100-0473bf56a052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bfbf8185ec34aacb47116b3866cc472'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:35.585768 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0cacd2fe-8e5e-4e12-9100-0473bf56a052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bfbf8185ec34aacb47116b3866cc472'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:35.592221 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1430/2857] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:26:35 2022] DELETE /identity/v3/users/5bfbf8185ec34aacb47116b3866cc472 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:35.765474 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f859f-e82c-4e78-b1c3-4219fc5a3728 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:35.775931 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f859f-e82c-4e78-b1c3-4219fc5a3728 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:35.833425 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cbb0de-23ea-4628-adf7-d33f86aadb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:35.834284 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbb0de-23ea-4628-adf7-d33f86aadb7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:35.834575 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbb0de-23ea-4628-adf7-d33f86aadb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:35.834857 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbb0de-23ea-4628-adf7-d33f86aadb7c admin admin] PATH_INFO: `/v3/projects/3ae9c262531e496286a1fcad0ce8e633` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:35.836065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cbb0de-23ea-4628-adf7-d33f86aadb7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ae9c262531e496286a1fcad0ce8e633)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:35.837989 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cbb0de-23ea-4628-adf7-d33f86aadb7c admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:35.882524 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-70cbb0de-23ea-4628-adf7-d33f86aadb7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ae9c262531e496286a1fcad0ce8e633', 'request_id': 'req-70cbb0de-23ea-4628-adf7-d33f86aadb7c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:35.888634 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-70cbb0de-23ea-4628-adf7-d33f86aadb7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ae9c262531e496286a1fcad0ce8e633', 'request_id': 'req-70cbb0de-23ea-4628-adf7-d33f86aadb7c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:35.895771 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-70cbb0de-23ea-4628-adf7-d33f86aadb7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ae9c262531e496286a1fcad0ce8e633', 'request_id': 'req-70cbb0de-23ea-4628-adf7-d33f86aadb7c'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:35.906614 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-70cbb0de-23ea-4628-adf7-d33f86aadb7c admin admin] The token cache is being invalidate because project 3ae9c262531e496286a1fcad0ce8e633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:26:35.906906 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-70cbb0de-23ea-4628-adf7-d33f86aadb7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:35.907555 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-70cbb0de-23ea-4628-adf7-d33f86aadb7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:35.909106 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1428/2858] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:26:35 2022] DELETE /identity/v3/projects/3ae9c262531e496286a1fcad0ce8e633 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:35.913151 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cacd2fe-8e5e-4e12-9100-0473bf56a052 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:35.915036 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0cacd2fe-8e5e-4e12-9100-0473bf56a052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:35.928021 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cacd2fe-8e5e-4e12-9100-0473bf56a052 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:35.985433 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23569b-df92-4d61-9c67-d97390300402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:35.986315 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23569b-df92-4d61-9c67-d97390300402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:35.986671 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23569b-df92-4d61-9c67-d97390300402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:35.986903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23569b-df92-4d61-9c67-d97390300402 admin admin] PATH_INFO: `/v3/users/89b77e7d07894c6a9940732d035eeaaa` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:35.988163 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23569b-df92-4d61-9c67-d97390300402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89b77e7d07894c6a9940732d035eeaaa)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:35.990099 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23569b-df92-4d61-9c67-d97390300402 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:36.011742 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:36.037291 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0a23569b-df92-4d61-9c67-d97390300402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89b77e7d07894c6a9940732d035eeaaa'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:36.044775 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0a23569b-df92-4d61-9c67-d97390300402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89b77e7d07894c6a9940732d035eeaaa'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:36.051752 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0a23569b-df92-4d61-9c67-d97390300402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89b77e7d07894c6a9940732d035eeaaa'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:36.052410 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0a23569b-df92-4d61-9c67-d97390300402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89b77e7d07894c6a9940732d035eeaaa'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:36.064825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0a23569b-df92-4d61-9c67-d97390300402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89b77e7d07894c6a9940732d035eeaaa'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:36.071127 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1431/2859] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:26:35 2022] DELETE /identity/v3/users/89b77e7d07894c6a9940732d035eeaaa => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:36.241712 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cbb0de-23ea-4628-adf7-d33f86aadb7c admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:36.243300 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-70cbb0de-23ea-4628-adf7-d33f86aadb7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:36.256205 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cbb0de-23ea-4628-adf7-d33f86aadb7c admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:36.312278 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ed656-c62c-47f8-ba62-17bdf299c80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:36.313152 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ed656-c62c-47f8-ba62-17bdf299c80e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:36.313464 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ed656-c62c-47f8-ba62-17bdf299c80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:36.313773 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ed656-c62c-47f8-ba62-17bdf299c80e admin admin] PATH_INFO: `/v3/projects/88b56e01548c40dfabfe48ce189d7bc9` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:36.315097 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ed656-c62c-47f8-ba62-17bdf299c80e admin admin] RBAC: Authorizing `identity:delete_project(project_id=88b56e01548c40dfabfe48ce189d7bc9)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:36.317093 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ed656-c62c-47f8-ba62-17bdf299c80e admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:36.352612 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:36.365539 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d45ed656-c62c-47f8-ba62-17bdf299c80e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88b56e01548c40dfabfe48ce189d7bc9', 'request_id': 'req-d45ed656-c62c-47f8-ba62-17bdf299c80e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:36.371494 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d45ed656-c62c-47f8-ba62-17bdf299c80e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88b56e01548c40dfabfe48ce189d7bc9', 'request_id': 'req-d45ed656-c62c-47f8-ba62-17bdf299c80e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:36.378398 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d45ed656-c62c-47f8-ba62-17bdf299c80e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88b56e01548c40dfabfe48ce189d7bc9', 'request_id': 'req-d45ed656-c62c-47f8-ba62-17bdf299c80e'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:36.389955 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d45ed656-c62c-47f8-ba62-17bdf299c80e admin admin] The token cache is being invalidate because project 88b56e01548c40dfabfe48ce189d7bc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:26:36.390250 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d45ed656-c62c-47f8-ba62-17bdf299c80e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:36.390900 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-d45ed656-c62c-47f8-ba62-17bdf299c80e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:36.392475 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1429/2860] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:26:36 2022] DELETE /identity/v3/projects/88b56e01548c40dfabfe48ce189d7bc9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:36.397342 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839ab34-2a35-4171-9b2f-aff1b786089d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:36.397644 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839ab34-2a35-4171-9b2f-aff1b786089d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:36.397931 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839ab34-2a35-4171-9b2f-aff1b786089d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:36.415471 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7839ab34-2a35-4171-9b2f-aff1b786089d None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:36.416711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7839ab34-2a35-4171-9b2f-aff1b786089d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:36.475464 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1432/2861] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:36.483198 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb606304-4b1c-4d0e-a8f7-7f03488c79d5 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:36.483497 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb606304-4b1c-4d0e-a8f7-7f03488c79d5 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:36.483777 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb606304-4b1c-4d0e-a8f7-7f03488c79d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:36.502109 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-eb606304-4b1c-4d0e-a8f7-7f03488c79d5 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:36.503239 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-eb606304-4b1c-4d0e-a8f7-7f03488c79d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:36.511614 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1430/2862] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:36.515709 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839ab34-2a35-4171-9b2f-aff1b786089d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:36.525507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839ab34-2a35-4171-9b2f-aff1b786089d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:36.530189 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed20512-5aa4-4cfa-9899-a91c274ee7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:36.530842 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed20512-5aa4-4cfa-9899-a91c274ee7c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:36.531136 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed20512-5aa4-4cfa-9899-a91c274ee7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:36.531421 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed20512-5aa4-4cfa-9899-a91c274ee7c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:36.532186 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed20512-5aa4-4cfa-9899-a91c274ee7c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:26:36.532503 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed20512-5aa4-4cfa-9899-a91c274ee7c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:36.534108 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed20512-5aa4-4cfa-9899-a91c274ee7c7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:36.545583 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1433/2863] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:26:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:26:36.549724 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb606304-4b1c-4d0e-a8f7-7f03488c79d5 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:36.559290 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb606304-4b1c-4d0e-a8f7-7f03488c79d5 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:36.563871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363c2cb-6284-42de-aea2-245cb93c8f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:36.564632 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363c2cb-6284-42de-aea2-245cb93c8f8b admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:36.564925 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363c2cb-6284-42de-aea2-245cb93c8f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:36.565207 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363c2cb-6284-42de-aea2-245cb93c8f8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:36.565891 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363c2cb-6284-42de-aea2-245cb93c8f8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:36.567773 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363c2cb-6284-42de-aea2-245cb93c8f8b admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:36.593205 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1431/2864] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:26:36 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:26:36.597292 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed20512-5aa4-4cfa-9899-a91c274ee7c7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:36.607128 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed20512-5aa4-4cfa-9899-a91c274ee7c7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:36.662153 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc456484-cfad-4f24-90be-a0118210b1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:36.662819 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc456484-cfad-4f24-90be-a0118210b1b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:36.663108 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc456484-cfad-4f24-90be-a0118210b1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:36.663390 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc456484-cfad-4f24-90be-a0118210b1b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:36.664108 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc456484-cfad-4f24-90be-a0118210b1b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:36.666051 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc456484-cfad-4f24-90be-a0118210b1b6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:36.685536 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1434/2865] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:26:36 2022] POST /identity/v3/users => generated 287 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:26:36.689549 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363c2cb-6284-42de-aea2-245cb93c8f8b admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:36.700508 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363c2cb-6284-42de-aea2-245cb93c8f8b admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:36.705112 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22eda2-954f-4b0d-8737-df18caa55189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:36.705764 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22eda2-954f-4b0d-8737-df18caa55189 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:36.706047 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22eda2-954f-4b0d-8737-df18caa55189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:36.706325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22eda2-954f-4b0d-8737-df18caa55189 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:36.707073 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22eda2-954f-4b0d-8737-df18caa55189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:36.708694 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a22eda2-954f-4b0d-8737-df18caa55189 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:36.718743 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1432/2866] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:26:36.722449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc456484-cfad-4f24-90be-a0118210b1b6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:36.732775 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc456484-cfad-4f24-90be-a0118210b1b6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:36.737370 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62bbf0d-d533-48b9-af06-d7a3679db6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:36.738019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62bbf0d-d533-48b9-af06-d7a3679db6c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:36.738308 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62bbf0d-d533-48b9-af06-d7a3679db6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:36.738589 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62bbf0d-d533-48b9-af06-d7a3679db6c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:36.739313 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62bbf0d-d533-48b9-af06-d7a3679db6c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:36.740936 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62bbf0d-d533-48b9-af06-d7a3679db6c1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:36.751200 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1435/2867] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:26:36.755062 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22eda2-954f-4b0d-8737-df18caa55189 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:36.764625 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22eda2-954f-4b0d-8737-df18caa55189 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:36.769121 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45866d-5114-4aca-ac6f-1149770e92a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:36.769974 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45866d-5114-4aca-ac6f-1149770e92a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:36.770254 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45866d-5114-4aca-ac6f-1149770e92a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:36.770528 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45866d-5114-4aca-ac6f-1149770e92a2 admin admin] PATH_INFO: `/v3/projects/42094d28a16d4cae87e4488bf3794d72/users/a32edeb93d1343fa80f3756589c7e851/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:36.784506 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45866d-5114-4aca-ac6f-1149770e92a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42094d28a16d4cae87e4488bf3794d72, user_id=a32edeb93d1343fa80f3756589c7e851, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:36.787148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45866d-5114-4aca-ac6f-1149770e92a2 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:36.800036 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1433/2868] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:26:36 2022] PUT /identity/v3/projects/42094d28a16d4cae87e4488bf3794d72/users/a32edeb93d1343fa80f3756589c7e851/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:36.811582 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aea14d-2c70-413a-97e6-1e71e3752aad None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:36.811884 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aea14d-2c70-413a-97e6-1e71e3752aad None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:36.812195 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aea14d-2c70-413a-97e6-1e71e3752aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:36.819032 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:36.841053 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-95aea14d-2c70-413a-97e6-1e71e3752aad None None] MFA Rules not processed for user `a32edeb93d1343fa80f3756589c7e851`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:36.842264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-95aea14d-2c70-413a-97e6-1e71e3752aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:36.905440 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1436/2869] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:26:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:26:36.916918 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45866d-5114-4aca-ac6f-1149770e92a2 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:36.918599 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-6c45866d-5114-4aca-ac6f-1149770e92a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:36.978713 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45866d-5114-4aca-ac6f-1149770e92a2 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:36.982332 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66847b47-25ae-432b-9e03-70d2a71b3a79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ee15f7716541a1a61443069d5cf4f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:36.982972 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66847b47-25ae-432b-9e03-70d2a71b3a79 service cinder] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:36.983261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66847b47-25ae-432b-9e03-70d2a71b3a79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:36.983541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66847b47-25ae-432b-9e03-70d2a71b3a79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:36.991969 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66847b47-25ae-432b-9e03-70d2a71b3a79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:36.994068 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66847b47-25ae-432b-9e03-70d2a71b3a79 service cinder] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:37.002928 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1434/2870] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:26:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:26:38.346070 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c136a7-cc63-476d-8e7c-628b8ce7bc49 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:38.346571 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c136a7-cc63-476d-8e7c-628b8ce7bc49 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:38.346692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c136a7-cc63-476d-8e7c-628b8ce7bc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:38.364580 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f8c136a7-cc63-476d-8e7c-628b8ce7bc49 None None] MFA Rules not processed for user `a32edeb93d1343fa80f3756589c7e851`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:38.365784 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f8c136a7-cc63-476d-8e7c-628b8ce7bc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:38.373279 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1437/2871] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:26:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:39.838786 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66847b47-25ae-432b-9e03-70d2a71b3a79 service cinder] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:39.840218 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-66847b47-25ae-432b-9e03-70d2a71b3a79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:39.897383 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:39.917748 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66847b47-25ae-432b-9e03-70d2a71b3a79 service cinder] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:39.922805 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c968b04-879a-4db6-a8de-a3cc27408b63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd5b2ba9e37041c78a99f1700a4970c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:39.923494 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c968b04-879a-4db6-a8de-a3cc27408b63 service placement] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:39.923786 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c968b04-879a-4db6-a8de-a3cc27408b63 service placement] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:39.924102 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c968b04-879a-4db6-a8de-a3cc27408b63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:39.925909 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-1c968b04-879a-4db6-a8de-a3cc27408b63 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:39.937977 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c968b04-879a-4db6-a8de-a3cc27408b63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:39.939578 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c968b04-879a-4db6-a8de-a3cc27408b63 service placement] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:39.956510 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1435/2872] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:26:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:26:45.618545 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c136a7-cc63-476d-8e7c-628b8ce7bc49 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:45.620524 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f8c136a7-cc63-476d-8e7c-628b8ce7bc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:45.771267 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c136a7-cc63-476d-8e7c-628b8ce7bc49 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:45.777716 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400f55f-599a-45b5-80da-67b79e088f02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:45.778482 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400f55f-599a-45b5-80da-67b79e088f02 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:45.778782 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400f55f-599a-45b5-80da-67b79e088f02 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:45.779297 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400f55f-599a-45b5-80da-67b79e088f02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:45.781175 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6400f55f-599a-45b5-80da-67b79e088f02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:45.794494 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400f55f-599a-45b5-80da-67b79e088f02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:45.796245 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400f55f-599a-45b5-80da-67b79e088f02 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:45.809060 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1438/2873] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:26:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:26:49.947668 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=8 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:50.440871 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c968b04-879a-4db6-a8de-a3cc27408b63 service placement] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:50.512567 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c968b04-879a-4db6-a8de-a3cc27408b63 service placement] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:50.518448 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae9a275-2868-4194-8ab6-ed541dfa1949 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac717fcd9fc1499db12cf852d45572a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:50.519122 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9a275-2868-4194-8ab6-ed541dfa1949 service nova] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:50.519412 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9a275-2868-4194-8ab6-ed541dfa1949 service nova] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:50.519688 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9a275-2868-4194-8ab6-ed541dfa1949 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:50.521607 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2ae9a275-2868-4194-8ab6-ed541dfa1949 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:50.548257 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae9a275-2868-4194-8ab6-ed541dfa1949 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:50.551000 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae9a275-2868-4194-8ab6-ed541dfa1949 service nova] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:50.562142 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1436/2874] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:26:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:26:55.804217 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=86 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:56.627932 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf1866-2883-4c3d-a8dc-cf67c0e5b26f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:56.628256 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf1866-2883-4c3d-a8dc-cf67c0e5b26f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:56.628545 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf1866-2883-4c3d-a8dc-cf67c0e5b26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:56.646280 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-03bf1866-2883-4c3d-a8dc-cf67c0e5b26f None None] MFA Rules not processed for user `94d824bffc2c4af299c537056bc21734`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:56.647488 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-03bf1866-2883-4c3d-a8dc-cf67c0e5b26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:56.702780 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1439/2875] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Jul 18 14:26:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:56.748780 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae9a275-2868-4194-8ab6-ed541dfa1949 service nova] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:56.750275 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2ae9a275-2868-4194-8ab6-ed541dfa1949 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:56.817387 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae9a275-2868-4194-8ab6-ed541dfa1949 service nova] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:56.822493 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf6b5d-72ad-450f-98e3-04cdb3b3ec5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbf513ee25374e49b3d8ce28843e393a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:56.823133 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf6b5d-72ad-450f-98e3-04cdb3b3ec5d service swift] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:56.823422 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf6b5d-72ad-450f-98e3-04cdb3b3ec5d service swift] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:56.823702 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf6b5d-72ad-450f-98e3-04cdb3b3ec5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:56.831598 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf6b5d-72ad-450f-98e3-04cdb3b3ec5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:56.833205 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf6b5d-72ad-450f-98e3-04cdb3b3ec5d service swift] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:56.841719 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1437/2876] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Jul 18 14:26:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:26:57.329855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656cc8a-3e45-459d-b504-d39ac1a8fa4e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:57.330357 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656cc8a-3e45-459d-b504-d39ac1a8fa4e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:57.330501 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656cc8a-3e45-459d-b504-d39ac1a8fa4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:57.347603 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0656cc8a-3e45-459d-b504-d39ac1a8fa4e None None] MFA Rules not processed for user `94d824bffc2c4af299c537056bc21734`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:57.348833 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0656cc8a-3e45-459d-b504-d39ac1a8fa4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:57.355884 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1440/2877] 10.222.0.94 () {58 vars in 967 bytes} [Mon Jul 18 14:26:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:57.367631 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf6b5d-72ad-450f-98e3-04cdb3b3ec5d service swift] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:57.378805 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf6b5d-72ad-450f-98e3-04cdb3b3ec5d service swift] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:57.384499 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0488d203-dcfe-462e-9544-3696eaf55dc8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbf513ee25374e49b3d8ce28843e393a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:57.385132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488d203-dcfe-462e-9544-3696eaf55dc8 service swift] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:57.385415 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488d203-dcfe-462e-9544-3696eaf55dc8 service swift] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:57.385692 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488d203-dcfe-462e-9544-3696eaf55dc8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:57.393728 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0488d203-dcfe-462e-9544-3696eaf55dc8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:57.395276 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0488d203-dcfe-462e-9544-3696eaf55dc8 service swift] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:57.403451 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1438/2878] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Jul 18 14:26:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:26:57.587658 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1475ff4-81de-4ccb-ab6d-25869aeb5927 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:57.587933 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1475ff4-81de-4ccb-ab6d-25869aeb5927 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:57.588246 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1475ff4-81de-4ccb-ab6d-25869aeb5927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:57.607360 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c1475ff4-81de-4ccb-ab6d-25869aeb5927 None None] MFA Rules not processed for user `94d824bffc2c4af299c537056bc21734`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:57.608606 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c1475ff4-81de-4ccb-ab6d-25869aeb5927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:57.615714 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1441/2879] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Jul 18 14:26:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:57.627778 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0488d203-dcfe-462e-9544-3696eaf55dc8 service swift] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:57.638802 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0488d203-dcfe-462e-9544-3696eaf55dc8 service swift] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:57.643792 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feacbc1-e78d-4d33-a196-f7ca048346ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbf513ee25374e49b3d8ce28843e393a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:57.644455 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feacbc1-e78d-4d33-a196-f7ca048346ce service swift] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:57.644743 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feacbc1-e78d-4d33-a196-f7ca048346ce service swift] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:57.645023 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feacbc1-e78d-4d33-a196-f7ca048346ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:57.653692 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feacbc1-e78d-4d33-a196-f7ca048346ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:57.655246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feacbc1-e78d-4d33-a196-f7ca048346ce service swift] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:57.663243 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1439/2880] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Jul 18 14:26:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:26:57.977361 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1cd2ab-4f4c-4514-b13c-222233001807 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:57.977660 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1cd2ab-4f4c-4514-b13c-222233001807 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:57.977942 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1cd2ab-4f4c-4514-b13c-222233001807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:57.995425 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-af1cd2ab-4f4c-4514-b13c-222233001807 None None] MFA Rules not processed for user `94d824bffc2c4af299c537056bc21734`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:57.996653 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-af1cd2ab-4f4c-4514-b13c-222233001807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:58.003717 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1442/2881] 10.222.0.94 () {58 vars in 967 bytes} [Mon Jul 18 14:26:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:58.016022 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feacbc1-e78d-4d33-a196-f7ca048346ce service swift] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:58.027351 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feacbc1-e78d-4d33-a196-f7ca048346ce service swift] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:58.032454 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd113ba-8405-4e17-9ce7-513d3ccc3ed9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbf513ee25374e49b3d8ce28843e393a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:58.033094 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd113ba-8405-4e17-9ce7-513d3ccc3ed9 service swift] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:58.033389 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd113ba-8405-4e17-9ce7-513d3ccc3ed9 service swift] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:58.033671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd113ba-8405-4e17-9ce7-513d3ccc3ed9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:58.041796 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd113ba-8405-4e17-9ce7-513d3ccc3ed9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:58.043402 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd113ba-8405-4e17-9ce7-513d3ccc3ed9 service swift] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:58.051502 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1440/2882] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Jul 18 14:26:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:27:00.092955 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1cd2ab-4f4c-4514-b13c-222233001807 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:00.171825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1cd2ab-4f4c-4514-b13c-222233001807 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:00.176449 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1502e-ac60-49fd-8692-cf823e0f0c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:00.177316 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1502e-ac60-49fd-8692-cf823e0f0c10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:00.177604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1502e-ac60-49fd-8692-cf823e0f0c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:00.177889 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1502e-ac60-49fd-8692-cf823e0f0c10 admin admin] PATH_INFO: `/v3/users/a32edeb93d1343fa80f3756589c7e851` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:00.179084 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1502e-ac60-49fd-8692-cf823e0f0c10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a32edeb93d1343fa80f3756589c7e851)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:00.181092 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1502e-ac60-49fd-8692-cf823e0f0c10 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:00.227132 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-37f1502e-ac60-49fd-8692-cf823e0f0c10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a32edeb93d1343fa80f3756589c7e851'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:00.233979 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-37f1502e-ac60-49fd-8692-cf823e0f0c10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a32edeb93d1343fa80f3756589c7e851'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:00.240981 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-37f1502e-ac60-49fd-8692-cf823e0f0c10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a32edeb93d1343fa80f3756589c7e851'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:00.241636 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-37f1502e-ac60-49fd-8692-cf823e0f0c10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a32edeb93d1343fa80f3756589c7e851'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:00.247029 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:00.254860 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-37f1502e-ac60-49fd-8692-cf823e0f0c10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a32edeb93d1343fa80f3756589c7e851'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:00.261621 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1443/2883] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:27:00 2022] DELETE /identity/v3/users/a32edeb93d1343fa80f3756589c7e851 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:00.266058 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0c917-6b79-4a1a-ad62-5d1cc5cd296f None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:00.266533 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0c917-6b79-4a1a-ad62-5d1cc5cd296f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:00.266660 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0c917-6b79-4a1a-ad62-5d1cc5cd296f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:00.303494 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-93e0c917-6b79-4a1a-ad62-5d1cc5cd296f None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:00.304723 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-93e0c917-6b79-4a1a-ad62-5d1cc5cd296f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:00.321937 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=100 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:00.364667 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1441/2884] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:00.374836 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1502e-ac60-49fd-8692-cf823e0f0c10 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:00.376304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-37f1502e-ac60-49fd-8692-cf823e0f0c10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:00.454414 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1502e-ac60-49fd-8692-cf823e0f0c10 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:00.458141 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a46edf-8dea-49f2-9024-2935afaa08e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:00.458792 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a46edf-8dea-49f2-9024-2935afaa08e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:00.459089 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a46edf-8dea-49f2-9024-2935afaa08e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:00.459375 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a46edf-8dea-49f2-9024-2935afaa08e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:00.468594 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a46edf-8dea-49f2-9024-2935afaa08e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:00.470081 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a46edf-8dea-49f2-9024-2935afaa08e9 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:00.479508 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1444/2885] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:00.649624 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0c917-6b79-4a1a-ad62-5d1cc5cd296f None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:00.651066 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-93e0c917-6b79-4a1a-ad62-5d1cc5cd296f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:00.663246 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0c917-6b79-4a1a-ad62-5d1cc5cd296f None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:00.667908 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef81c38-2201-4661-aea9-a717d0345e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:00.668820 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef81c38-2201-4661-aea9-a717d0345e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:00.669110 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef81c38-2201-4661-aea9-a717d0345e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:00.669391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef81c38-2201-4661-aea9-a717d0345e93 admin admin] PATH_INFO: `/v3/projects/42094d28a16d4cae87e4488bf3794d72` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:00.670589 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef81c38-2201-4661-aea9-a717d0345e93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42094d28a16d4cae87e4488bf3794d72)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:00.672536 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef81c38-2201-4661-aea9-a717d0345e93 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:00.719505 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bef81c38-2201-4661-aea9-a717d0345e93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42094d28a16d4cae87e4488bf3794d72', 'request_id': 'req-bef81c38-2201-4661-aea9-a717d0345e93'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:00.724959 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bef81c38-2201-4661-aea9-a717d0345e93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42094d28a16d4cae87e4488bf3794d72', 'request_id': 'req-bef81c38-2201-4661-aea9-a717d0345e93'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:00.733409 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bef81c38-2201-4661-aea9-a717d0345e93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42094d28a16d4cae87e4488bf3794d72', 'request_id': 'req-bef81c38-2201-4661-aea9-a717d0345e93'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:00.745150 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bef81c38-2201-4661-aea9-a717d0345e93 admin admin] The token cache is being invalidate because project 42094d28a16d4cae87e4488bf3794d72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:27:00.745442 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bef81c38-2201-4661-aea9-a717d0345e93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:00.746090 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-bef81c38-2201-4661-aea9-a717d0345e93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:00.747669 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1442/2886] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:27:00 2022] DELETE /identity/v3/projects/42094d28a16d4cae87e4488bf3794d72 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:00.752558 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d03d8-ffb2-4637-99a8-4bb24563771e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:00.752941 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d03d8-ffb2-4637-99a8-4bb24563771e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:00.753229 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d03d8-ffb2-4637-99a8-4bb24563771e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:00.799025 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5a8d03d8-ffb2-4637-99a8-4bb24563771e None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:00.800398 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5a8d03d8-ffb2-4637-99a8-4bb24563771e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:00.861166 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1445/2887] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:27:00.870357 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fcba3-a4e5-4c85-8e13-618565fe38d4 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:00.870656 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fcba3-a4e5-4c85-8e13-618565fe38d4 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:00.870942 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fcba3-a4e5-4c85-8e13-618565fe38d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:00.888962 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c71fcba3-a4e5-4c85-8e13-618565fe38d4 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:00.890155 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c71fcba3-a4e5-4c85-8e13-618565fe38d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:00.897991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1443/2888] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:00.902113 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d03d8-ffb2-4637-99a8-4bb24563771e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:00.911857 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d03d8-ffb2-4637-99a8-4bb24563771e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:00.917025 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda06e90-d08c-4552-b2a9-427a1a3bfdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:00.917678 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda06e90-d08c-4552-b2a9-427a1a3bfdec admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:00.917966 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda06e90-d08c-4552-b2a9-427a1a3bfdec admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:00.918245 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda06e90-d08c-4552-b2a9-427a1a3bfdec admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:00.918968 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda06e90-d08c-4552-b2a9-427a1a3bfdec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:27:00.919279 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda06e90-d08c-4552-b2a9-427a1a3bfdec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:00.920939 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda06e90-d08c-4552-b2a9-427a1a3bfdec admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:00.932977 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1446/2889] 10.222.0.94 () {58 vars in 1152 bytes} [Mon Jul 18 14:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:27:00.937179 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fcba3-a4e5-4c85-8e13-618565fe38d4 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:00.946877 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71fcba3-a4e5-4c85-8e13-618565fe38d4 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:00.951680 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cdf63-3e5d-400b-84bc-90ac46c965d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:00.952374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cdf63-3e5d-400b-84bc-90ac46c965d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:00.952668 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cdf63-3e5d-400b-84bc-90ac46c965d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:00.952954 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cdf63-3e5d-400b-84bc-90ac46c965d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:00.953646 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473cdf63-3e5d-400b-84bc-90ac46c965d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:00.955554 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473cdf63-3e5d-400b-84bc-90ac46c965d9 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:00.980905 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1444/2890] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:27:00 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:00.985101 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda06e90-d08c-4552-b2a9-427a1a3bfdec admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:00.995511 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda06e90-d08c-4552-b2a9-427a1a3bfdec admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:01.024971 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:01.052082 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332bbd1-1e31-4504-bddb-947332814ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:01.052748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332bbd1-1e31-4504-bddb-947332814ac6 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:01.053041 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332bbd1-1e31-4504-bddb-947332814ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:01.053326 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332bbd1-1e31-4504-bddb-947332814ac6 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:01.054030 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332bbd1-1e31-4504-bddb-947332814ac6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:01.056615 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332bbd1-1e31-4504-bddb-947332814ac6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:01.075082 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1447/2891] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:27:00 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:27:01.079525 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cdf63-3e5d-400b-84bc-90ac46c965d9 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:01.090260 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cdf63-3e5d-400b-84bc-90ac46c965d9 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:01.094906 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4b867-7a45-45b4-8cb7-1bd99bb275ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:01.095560 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4b867-7a45-45b4-8cb7-1bd99bb275ac admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:01.095850 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4b867-7a45-45b4-8cb7-1bd99bb275ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:01.096159 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4b867-7a45-45b4-8cb7-1bd99bb275ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:01.096890 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d4b867-7a45-45b4-8cb7-1bd99bb275ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:01.098491 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d4b867-7a45-45b4-8cb7-1bd99bb275ac admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:01.109209 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1445/2892] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:01.113007 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332bbd1-1e31-4504-bddb-947332814ac6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:01.123010 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332bbd1-1e31-4504-bddb-947332814ac6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:01.127692 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1231669-541e-4bd5-9729-8a50c9a2475d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:01.128411 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1231669-541e-4bd5-9729-8a50c9a2475d admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:01.128702 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1231669-541e-4bd5-9729-8a50c9a2475d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:01.128985 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1231669-541e-4bd5-9729-8a50c9a2475d admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:01.129722 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1231669-541e-4bd5-9729-8a50c9a2475d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:01.131421 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1231669-541e-4bd5-9729-8a50c9a2475d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:01.141484 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1448/2893] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:01.145515 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4b867-7a45-45b4-8cb7-1bd99bb275ac admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:01.155836 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4b867-7a45-45b4-8cb7-1bd99bb275ac admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:01.160518 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a21da19-1425-4875-bf5b-7070dd62cdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:01.161385 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21da19-1425-4875-bf5b-7070dd62cdfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:01.161671 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21da19-1425-4875-bf5b-7070dd62cdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:01.161951 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21da19-1425-4875-bf5b-7070dd62cdfb admin admin] PATH_INFO: `/v3/projects/0c3ec8c8d2f84ab1a84a1f64fd8a66c7/users/caa872b8a1184533b7b57f0426d2e34c/roles/1399a838106b4f9bb5fa1c942cc3ef22` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:01.176613 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a21da19-1425-4875-bf5b-7070dd62cdfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c3ec8c8d2f84ab1a84a1f64fd8a66c7, user_id=caa872b8a1184533b7b57f0426d2e34c, role_id=1399a838106b4f9bb5fa1c942cc3ef22)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:01.179261 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a21da19-1425-4875-bf5b-7070dd62cdfb admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:01.191935 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1446/2894] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:01 2022] PUT /identity/v3/projects/0c3ec8c8d2f84ab1a84a1f64fd8a66c7/users/caa872b8a1184533b7b57f0426d2e34c/roles/1399a838106b4f9bb5fa1c942cc3ef22 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:01.197150 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b18e73-53bc-46f2-9dfc-29f056e234a4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:01.197480 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b18e73-53bc-46f2-9dfc-29f056e234a4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:01.197768 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b18e73-53bc-46f2-9dfc-29f056e234a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:01.216724 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-62b18e73-53bc-46f2-9dfc-29f056e234a4 None None] MFA Rules not processed for user `7a10bd46534e415080cc3fb7cb29bb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:01.217979 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-62b18e73-53bc-46f2-9dfc-29f056e234a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:01.397704 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1449/2895] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:27:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:27:01.407902 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a21da19-1425-4875-bf5b-7070dd62cdfb admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:01.409683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-3a21da19-1425-4875-bf5b-7070dd62cdfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:01.455317 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:01.469449 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a21da19-1425-4875-bf5b-7070dd62cdfb admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:01.473089 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6348e-5207-4238-99b2-611320c20304 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:01.473734 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6348e-5207-4238-99b2-611320c20304 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:01.474025 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6348e-5207-4238-99b2-611320c20304 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:01.474308 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6348e-5207-4238-99b2-611320c20304 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:01.483321 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6348e-5207-4238-99b2-611320c20304 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:01.484773 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6348e-5207-4238-99b2-611320c20304 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:01.494187 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1447/2896] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:05.455773 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5531da4-2831-498e-b204-55bfbabf6414 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:05.456301 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5531da4-2831-498e-b204-55bfbabf6414 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:05.456527 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5531da4-2831-498e-b204-55bfbabf6414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:05.485364 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e5531da4-2831-498e-b204-55bfbabf6414 None None] MFA Rules not processed for user `caa872b8a1184533b7b57f0426d2e34c`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:05.486673 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e5531da4-2831-498e-b204-55bfbabf6414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:05.561269 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1450/2897] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:27:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:05.566126 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6348e-5207-4238-99b2-611320c20304 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:05.577708 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6348e-5207-4238-99b2-611320c20304 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:05.582462 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e2387-63d1-49cd-88d9-48c4fbd60d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:05.583143 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e2387-63d1-49cd-88d9-48c4fbd60d3f admin admin] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:05.583435 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e2387-63d1-49cd-88d9-48c4fbd60d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:05.583719 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e2387-63d1-49cd-88d9-48c4fbd60d3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:05.584531 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e2387-63d1-49cd-88d9-48c4fbd60d3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:05.586510 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e2387-63d1-49cd-88d9-48c4fbd60d3f admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:05.618493 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1448/2898] 10.222.0.94 () {60 vars in 1152 bytes} [Mon Jul 18 14:27:05 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:27:05.622919 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5531da4-2831-498e-b204-55bfbabf6414 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:05.637790 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5531da4-2831-498e-b204-55bfbabf6414 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:05.701254 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:05.727765 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2965d-627e-4a7f-974e-ce113760e507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:05.728532 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2965d-627e-4a7f-974e-ce113760e507 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:05.728825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2965d-627e-4a7f-974e-ce113760e507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:05.729108 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2965d-627e-4a7f-974e-ce113760e507 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:05.729810 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2965d-627e-4a7f-974e-ce113760e507 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:05.731748 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2965d-627e-4a7f-974e-ce113760e507 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:05.752507 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1451/2899] 10.222.0.94 () {60 vars in 1143 bytes} [Mon Jul 18 14:27:05 2022] POST /identity/v3/users => generated 299 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:27:05.756548 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e2387-63d1-49cd-88d9-48c4fbd60d3f admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:05.767698 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e2387-63d1-49cd-88d9-48c4fbd60d3f admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:05.772418 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3cdd19-9985-4b6b-9cab-d82ab957c6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:05.773078 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cdd19-9985-4b6b-9cab-d82ab957c6aa admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:05.773369 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cdd19-9985-4b6b-9cab-d82ab957c6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:05.773648 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cdd19-9985-4b6b-9cab-d82ab957c6aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:05.774374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3cdd19-9985-4b6b-9cab-d82ab957c6aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:05.775985 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3cdd19-9985-4b6b-9cab-d82ab957c6aa admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:05.786347 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1449/2900] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:05.790373 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2965d-627e-4a7f-974e-ce113760e507 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:05.800273 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2965d-627e-4a7f-974e-ce113760e507 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:05.808296 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699b234-4146-4244-9e0d-dd9062a3425b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:05.809141 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699b234-4146-4244-9e0d-dd9062a3425b admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:05.809432 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699b234-4146-4244-9e0d-dd9062a3425b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:05.809711 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699b234-4146-4244-9e0d-dd9062a3425b admin admin] PATH_INFO: `/v3/projects/347f076ba63f4ed69aecd3764c302f5b/users/8fda79162e8e418698513f16d25884eb/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:05.825325 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699b234-4146-4244-9e0d-dd9062a3425b admin admin] RBAC: Authorizing `identity:create_grant(project_id=347f076ba63f4ed69aecd3764c302f5b, user_id=8fda79162e8e418698513f16d25884eb, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:05.827993 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699b234-4146-4244-9e0d-dd9062a3425b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:05.841100 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1452/2901] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:05 2022] PUT /identity/v3/projects/347f076ba63f4ed69aecd3764c302f5b/users/8fda79162e8e418698513f16d25884eb/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:27:05.845072 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3cdd19-9985-4b6b-9cab-d82ab957c6aa admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:05.855152 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3cdd19-9985-4b6b-9cab-d82ab957c6aa admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:05.911919 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7fcbf-05a8-4ff2-adbb-7847fb0111a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:05.912605 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7fcbf-05a8-4ff2-adbb-7847fb0111a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:05.912890 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7fcbf-05a8-4ff2-adbb-7847fb0111a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:05.913165 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7fcbf-05a8-4ff2-adbb-7847fb0111a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:05.913876 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7fcbf-05a8-4ff2-adbb-7847fb0111a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:05.915481 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7fcbf-05a8-4ff2-adbb-7847fb0111a1 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:05.925030 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1450/2902] 10.222.0.94 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:27:05.929144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699b234-4146-4244-9e0d-dd9062a3425b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:05.939761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699b234-4146-4244-9e0d-dd9062a3425b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:05.944517 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e4431-d6f2-420e-94b3-eb80d3a98059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:05.945399 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e4431-d6f2-420e-94b3-eb80d3a98059 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:05.946581 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e4431-d6f2-420e-94b3-eb80d3a98059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:05.946880 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e4431-d6f2-420e-94b3-eb80d3a98059 admin admin] PATH_INFO: `/v3/projects/347f076ba63f4ed69aecd3764c302f5b/users/8fda79162e8e418698513f16d25884eb/roles/c60a99de086f45c6820f7872a6b43feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:05.948924 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e4431-d6f2-420e-94b3-eb80d3a98059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=347f076ba63f4ed69aecd3764c302f5b, user_id=8fda79162e8e418698513f16d25884eb, role_id=c60a99de086f45c6820f7872a6b43feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:05.951555 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e4431-d6f2-420e-94b3-eb80d3a98059 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:05.961909 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1453/2903] 10.222.0.94 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:05 2022] PUT /identity/v3/projects/347f076ba63f4ed69aecd3764c302f5b/users/8fda79162e8e418698513f16d25884eb/roles/c60a99de086f45c6820f7872a6b43feb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:10.419374 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f7b5a-5de6-4d1f-bc8b-ce19a6ddfb0e None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:10.419922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f7b5a-5de6-4d1f-bc8b-ce19a6ddfb0e None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:10.419922 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f7b5a-5de6-4d1f-bc8b-ce19a6ddfb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:10.450153 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-de1f7b5a-5de6-4d1f-bc8b-ce19a6ddfb0e None None] MFA Rules not processed for user `8fda79162e8e418698513f16d25884eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:10.451373 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-de1f7b5a-5de6-4d1f-bc8b-ce19a6ddfb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:10.543496 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1451/2904] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:27:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:10.556615 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e4431-d6f2-420e-94b3-eb80d3a98059 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:10.558349 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-392e4431-d6f2-420e-94b3-eb80d3a98059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:10.616761 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e4431-d6f2-420e-94b3-eb80d3a98059 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:10.621087 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4512e6b-f230-434c-aa6d-aa1804deb651 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '503a8228d4754c309fc3f365f4cfc61b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:10.621831 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4512e6b-f230-434c-aa6d-aa1804deb651 service glance] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:10.622193 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4512e6b-f230-434c-aa6d-aa1804deb651 service glance] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:10.622546 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4512e6b-f230-434c-aa6d-aa1804deb651 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:10.683147 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4512e6b-f230-434c-aa6d-aa1804deb651 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:10.684651 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4512e6b-f230-434c-aa6d-aa1804deb651 service glance] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:10.694497 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1454/2905] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:10.799919 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040473da-ca9d-406d-8bf2-397a847ac35f None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:10.800436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040473da-ca9d-406d-8bf2-397a847ac35f None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:10.800789 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040473da-ca9d-406d-8bf2-397a847ac35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:10.820134 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-040473da-ca9d-406d-8bf2-397a847ac35f None None] MFA Rules not processed for user `94d824bffc2c4af299c537056bc21734`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:10.821755 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-040473da-ca9d-406d-8bf2-397a847ac35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:10.853016 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:10.882339 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1452/2906] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Jul 18 14:27:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:10.894193 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4512e6b-f230-434c-aa6d-aa1804deb651 service glance] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:10.895598 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a4512e6b-f230-434c-aa6d-aa1804deb651 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:10.948479 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:10.971552 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4512e6b-f230-434c-aa6d-aa1804deb651 service glance] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:10.976653 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf8320-ed52-4e0e-a7bc-3b4bb29232aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbf513ee25374e49b3d8ce28843e393a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:10.977888 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf8320-ed52-4e0e-a7bc-3b4bb29232aa service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:10.978175 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf8320-ed52-4e0e-a7bc-3b4bb29232aa service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:10.978454 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf8320-ed52-4e0e-a7bc-3b4bb29232aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:10.987061 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf8320-ed52-4e0e-a7bc-3b4bb29232aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:10.988715 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf8320-ed52-4e0e-a7bc-3b4bb29232aa service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:10.996720 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1455/2907] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Jul 18 14:27:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:27:19.312810 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e03db5-03c4-488f-9b02-87deb4ab1b35 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:19.313295 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e03db5-03c4-488f-9b02-87deb4ab1b35 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:19.313420 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e03db5-03c4-488f-9b02-87deb4ab1b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:19.330764 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-70e03db5-03c4-488f-9b02-87deb4ab1b35 None None] MFA Rules not processed for user `caa872b8a1184533b7b57f0426d2e34c`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:19.331971 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-70e03db5-03c4-488f-9b02-87deb4ab1b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:19.339282 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1453/2908] 10.222.0.94 () {58 vars in 957 bytes} [Mon Jul 18 14:27:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:19.590170 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf8320-ed52-4e0e-a7bc-3b4bb29232aa service swift] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:19.591589 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4caf8320-ed52-4e0e-a7bc-3b4bb29232aa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:19.645615 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf8320-ed52-4e0e-a7bc-3b4bb29232aa service swift] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:19.649623 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef629489-d90c-4d61-9163-262a5eb7a53c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:19.650252 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef629489-d90c-4d61-9163-262a5eb7a53c service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:19.650534 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef629489-d90c-4d61-9163-262a5eb7a53c service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:19.650807 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef629489-d90c-4d61-9163-262a5eb7a53c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:19.652564 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ef629489-d90c-4d61-9163-262a5eb7a53c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:19.661686 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef629489-d90c-4d61-9163-262a5eb7a53c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:19.663093 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef629489-d90c-4d61-9163-262a5eb7a53c service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:19.670917 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1456/2909] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:21.216312 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ef53b-971b-4fed-91d9-d66db4901be7 None None] REQUEST_METHOD: `POST` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:21.216822 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ef53b-971b-4fed-91d9-d66db4901be7 None None] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:21.216912 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ef53b-971b-4fed-91d9-d66db4901be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:21.235086 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.auth.core [None req-c17ef53b-971b-4fed-91d9-d66db4901be7 None None] MFA Rules not processed for user `94d824bffc2c4af299c537056bc21734`. Rule list: `[]` (Enabled: `True`). {{(pid=80158) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:21.236370 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c17ef53b-971b-4fed-91d9-d66db4901be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:21.244154 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1454/2910] 10.222.0.94 () {58 vars in 1009 bytes} [Mon Jul 18 14:27:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:21.261960 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef629489-d90c-4d61-9163-262a5eb7a53c service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:21.280903 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef629489-d90c-4d61-9163-262a5eb7a53c service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:21.289899 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cf27d-ba59-4a7e-b9c5-89acc96cf420 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbf513ee25374e49b3d8ce28843e393a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:21.290809 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cf27d-ba59-4a7e-b9c5-89acc96cf420 service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:21.291207 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cf27d-ba59-4a7e-b9c5-89acc96cf420 service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:21.291588 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cf27d-ba59-4a7e-b9c5-89acc96cf420 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:21.302943 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cf27d-ba59-4a7e-b9c5-89acc96cf420 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:21.304604 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cf27d-ba59-4a7e-b9c5-89acc96cf420 service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:21.313718 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1457/2911] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Jul 18 14:27:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:27:31.229903 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=16 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:31.310359 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=42 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:42.849721 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ef53b-971b-4fed-91d9-d66db4901be7 None None] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:42.851126 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-c17ef53b-971b-4fed-91d9-d66db4901be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:42.943051 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ef53b-971b-4fed-91d9-d66db4901be7 None None] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:42.948580 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b27aa-c742-4d62-8b52-09135d419c71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd5b2ba9e37041c78a99f1700a4970c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:42.949236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b27aa-c742-4d62-8b52-09135d419c71 service placement] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:42.949525 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b27aa-c742-4d62-8b52-09135d419c71 service placement] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:42.949808 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b27aa-c742-4d62-8b52-09135d419c71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:42.951571 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-8f3b27aa-c742-4d62-8b52-09135d419c71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:42.963235 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b27aa-c742-4d62-8b52-09135d419c71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:42.964890 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b27aa-c742-4d62-8b52-09135d419c71 service placement] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:42.974732 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1455/2912] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:51.303899 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cf27d-ba59-4a7e-b9c5-89acc96cf420 service swift] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:51.313292 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cf27d-ba59-4a7e-b9c5-89acc96cf420 service swift] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:51.317434 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e1b78-ade0-4d47-97ae-50ccb99f5547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:51.318165 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e1b78-ade0-4d47-97ae-50ccb99f5547 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:51.318455 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e1b78-ade0-4d47-97ae-50ccb99f5547 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:51.318734 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e1b78-ade0-4d47-97ae-50ccb99f5547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:51.327847 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05e1b78-ade0-4d47-97ae-50ccb99f5547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:51.329304 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05e1b78-ade0-4d47-97ae-50ccb99f5547 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:51.338592 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1458/2913] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:52.970541 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=41 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:01.334514 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:55.329507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b27aa-c742-4d62-8b52-09135d419c71 service placement] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:55.339601 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b27aa-c742-4d62-8b52-09135d419c71 service placement] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:55.343217 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f592a-6fe0-463c-81bf-52705d30c514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:55.343858 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f592a-6fe0-463c-81bf-52705d30c514 service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:55.344170 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f592a-6fe0-463c-81bf-52705d30c514 service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:55.344451 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f592a-6fe0-463c-81bf-52705d30c514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:55.346210 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-158f592a-6fe0-463c-81bf-52705d30c514 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:55.424589 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f592a-6fe0-463c-81bf-52705d30c514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:55.426065 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f592a-6fe0-463c-81bf-52705d30c514 service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:55.437085 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1456/2914] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:28:56.716910 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e1b78-ade0-4d47-97ae-50ccb99f5547 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:56.725600 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e1b78-ade0-4d47-97ae-50ccb99f5547 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:56.729168 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b37f238-9b97-40ed-8e73-451370d5e93a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:56.729827 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37f238-9b97-40ed-8e73-451370d5e93a service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:56.730115 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37f238-9b97-40ed-8e73-451370d5e93a service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:56.730391 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37f238-9b97-40ed-8e73-451370d5e93a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:56.732173 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4b37f238-9b97-40ed-8e73-451370d5e93a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:56.745226 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b37f238-9b97-40ed-8e73-451370d5e93a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:56.746632 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b37f238-9b97-40ed-8e73-451370d5e93a service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:56.757787 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1459/2915] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:29:05.432110 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=34 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:29:06.752569 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:29:50.519606 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f592a-6fe0-463c-81bf-52705d30c514 service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:29:50.529044 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f592a-6fe0-463c-81bf-52705d30c514 service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:29:50.532683 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d97d8-a0d3-4520-8558-b6102b42e23d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14027a0e13fd4272bd7803e3131b6348', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:29:50.533325 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d97d8-a0d3-4520-8558-b6102b42e23d service neutron] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:29:50.533614 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d97d8-a0d3-4520-8558-b6102b42e23d service neutron] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:29:50.533895 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d97d8-a0d3-4520-8558-b6102b42e23d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:29:50.535647 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-d47d97d8-a0d3-4520-8558-b6102b42e23d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:29:50.545187 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d97d8-a0d3-4520-8558-b6102b42e23d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:29:50.546608 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d97d8-a0d3-4520-8558-b6102b42e23d service neutron] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:29:50.554753 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1457/2916] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:29:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:30:00.551132 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=6 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:30:20.116235 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe41a59-7366-4572-973f-41bfdb45e926 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:30:20.117203 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe41a59-7366-4572-973f-41bfdb45e926 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:30:20.117314 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe41a59-7366-4572-973f-41bfdb45e926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:30:20.174732 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cfe41a59-7366-4572-973f-41bfdb45e926 None None] MFA Rules not processed for user `94d824bffc2c4af299c537056bc21734`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:30:20.175934 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cfe41a59-7366-4572-973f-41bfdb45e926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:30:20.192938 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1460/2917] 10.222.0.94 () {58 vars in 967 bytes} [Mon Jul 18 14:30:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:30:20.233668 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d97d8-a0d3-4520-8558-b6102b42e23d service neutron] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:30:20.256589 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d97d8-a0d3-4520-8558-b6102b42e23d service neutron] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:30:20.261507 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6915c-2789-4231-8ded-fa84d9b4b466 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbf513ee25374e49b3d8ce28843e393a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:30:20.262148 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6915c-2789-4231-8ded-fa84d9b4b466 service swift] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:30:20.262436 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6915c-2789-4231-8ded-fa84d9b4b466 service swift] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:30:20.262718 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6915c-2789-4231-8ded-fa84d9b4b466 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:30:20.270678 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6915c-2789-4231-8ded-fa84d9b4b466 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:30:20.272295 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6915c-2789-4231-8ded-fa84d9b4b466 service swift] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:30:20.280860 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1458/2918] 10.222.0.94 () {60 vars in 1384 bytes} [Mon Jul 18 14:30:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:30:23.689493 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe41a59-7366-4572-973f-41bfdb45e926 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:30:23.703170 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe41a59-7366-4572-973f-41bfdb45e926 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:30:23.708458 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3a3fe-adec-4188-ba38-cf650d9efd2a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd5b2ba9e37041c78a99f1700a4970c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:30:23.709106 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3a3fe-adec-4188-ba38-cf650d9efd2a service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:30:23.709397 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3a3fe-adec-4188-ba38-cf650d9efd2a service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:30:23.709705 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3a3fe-adec-4188-ba38-cf650d9efd2a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:30:23.711475 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dfc3a3fe-adec-4188-ba38-cf650d9efd2a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:30:23.723109 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3a3fe-adec-4188-ba38-cf650d9efd2a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:30:23.724733 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3a3fe-adec-4188-ba38-cf650d9efd2a service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:30:23.735394 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1461/2919] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:30:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:30:26.808859 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6915c-2789-4231-8ded-fa84d9b4b466 service swift] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:30:26.821552 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6915c-2789-4231-8ded-fa84d9b4b466 service swift] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:30:26.832203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd8d64-1b69-4269-b5db-948884c99585 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd5b2ba9e37041c78a99f1700a4970c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:30:26.832203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd8d64-1b69-4269-b5db-948884c99585 service placement] REQUEST_METHOD: `GET` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:30:26.832203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd8d64-1b69-4269-b5db-948884c99585 service placement] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:30:26.832203 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd8d64-1b69-4269-b5db-948884c99585 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:30:26.832670 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-68bd8d64-1b69-4269-b5db-948884c99585 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:30:26.841991 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd8d64-1b69-4269-b5db-948884c99585 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:30:26.843624 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd8d64-1b69-4269-b5db-948884c99585 service placement] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:30:26.862659 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1459/2920] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:30:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:30:26.876838 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3a3fe-adec-4188-ba38-cf650d9efd2a service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:30:26.941884 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3a3fe-adec-4188-ba38-cf650d9efd2a service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:30:26.946800 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2a545-710c-4ecc-bb21-4cd07ef342f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:30:26.947732 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2a545-710c-4ecc-bb21-4cd07ef342f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:30:26.948089 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2a545-710c-4ecc-bb21-4cd07ef342f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:30:26.948469 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2a545-710c-4ecc-bb21-4cd07ef342f0 admin admin] PATH_INFO: `/v3/users/caa872b8a1184533b7b57f0426d2e34c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:30:26.950197 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2a545-710c-4ecc-bb21-4cd07ef342f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caa872b8a1184533b7b57f0426d2e34c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:30:26.951758 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2a545-710c-4ecc-bb21-4cd07ef342f0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:30:27.001830 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-75d2a545-710c-4ecc-bb21-4cd07ef342f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caa872b8a1184533b7b57f0426d2e34c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:30:27.008349 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-75d2a545-710c-4ecc-bb21-4cd07ef342f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caa872b8a1184533b7b57f0426d2e34c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:30:27.015218 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-75d2a545-710c-4ecc-bb21-4cd07ef342f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caa872b8a1184533b7b57f0426d2e34c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:30:27.015857 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-75d2a545-710c-4ecc-bb21-4cd07ef342f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caa872b8a1184533b7b57f0426d2e34c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:30:27.027845 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-75d2a545-710c-4ecc-bb21-4cd07ef342f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caa872b8a1184533b7b57f0426d2e34c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:30:27.033713 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1462/2921] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:30:26 2022] DELETE /identity/v3/users/caa872b8a1184533b7b57f0426d2e34c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:30:27.211927 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd8d64-1b69-4269-b5db-948884c99585 service placement] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:30:27.222652 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd8d64-1b69-4269-b5db-948884c99585 service placement] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:30:27.280530 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a56b2-650f-4bb2-99bb-73973ad6d178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:30:27.281397 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a56b2-650f-4bb2-99bb-73973ad6d178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:30:27.281690 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a56b2-650f-4bb2-99bb-73973ad6d178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:30:27.281974 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a56b2-650f-4bb2-99bb-73973ad6d178 admin admin] PATH_INFO: `/v3/projects/0c3ec8c8d2f84ab1a84a1f64fd8a66c7` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:30:27.283169 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a56b2-650f-4bb2-99bb-73973ad6d178 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c3ec8c8d2f84ab1a84a1f64fd8a66c7)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:30:27.285119 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a56b2-650f-4bb2-99bb-73973ad6d178 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:30:27.332236 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-2b0a56b2-650f-4bb2-99bb-73973ad6d178 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c3ec8c8d2f84ab1a84a1f64fd8a66c7', 'request_id': 'req-2b0a56b2-650f-4bb2-99bb-73973ad6d178'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:30:27.339301 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-2b0a56b2-650f-4bb2-99bb-73973ad6d178 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c3ec8c8d2f84ab1a84a1f64fd8a66c7', 'request_id': 'req-2b0a56b2-650f-4bb2-99bb-73973ad6d178'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:30:27.346487 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-2b0a56b2-650f-4bb2-99bb-73973ad6d178 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c3ec8c8d2f84ab1a84a1f64fd8a66c7', 'request_id': 'req-2b0a56b2-650f-4bb2-99bb-73973ad6d178'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:30:27.357472 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-2b0a56b2-650f-4bb2-99bb-73973ad6d178 admin admin] The token cache is being invalidate because project 0c3ec8c8d2f84ab1a84a1f64fd8a66c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:30:27.357765 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-2b0a56b2-650f-4bb2-99bb-73973ad6d178 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:30:27.358391 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-2b0a56b2-650f-4bb2-99bb-73973ad6d178 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:30:27.359982 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1460/2922] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:30:27 2022] DELETE /identity/v3/projects/0c3ec8c8d2f84ab1a84a1f64fd8a66c7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:30:27.363847 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2a545-710c-4ecc-bb21-4cd07ef342f0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:30:27.365264 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-75d2a545-710c-4ecc-bb21-4cd07ef342f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:30:27.377853 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2a545-710c-4ecc-bb21-4cd07ef342f0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:30:27.390745 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:30:27.436108 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bc013-0d13-4566-96c7-10c69af21469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:30:27.436980 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bc013-0d13-4566-96c7-10c69af21469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:30:27.437273 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bc013-0d13-4566-96c7-10c69af21469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:30:27.437557 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129bc013-0d13-4566-96c7-10c69af21469 admin admin] PATH_INFO: `/v3/users/8fda79162e8e418698513f16d25884eb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:30:27.439314 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129bc013-0d13-4566-96c7-10c69af21469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fda79162e8e418698513f16d25884eb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:30:27.441283 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129bc013-0d13-4566-96c7-10c69af21469 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:30:27.488078 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-129bc013-0d13-4566-96c7-10c69af21469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fda79162e8e418698513f16d25884eb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:30:27.495825 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-129bc013-0d13-4566-96c7-10c69af21469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fda79162e8e418698513f16d25884eb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:30:27.502855 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-129bc013-0d13-4566-96c7-10c69af21469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fda79162e8e418698513f16d25884eb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:30:27.503510 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-129bc013-0d13-4566-96c7-10c69af21469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fda79162e8e418698513f16d25884eb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:30:27.516073 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-129bc013-0d13-4566-96c7-10c69af21469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fda79162e8e418698513f16d25884eb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:30:27.522019 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1463/2923] 10.222.0.94 () {58 vars in 1223 bytes} [Mon Jul 18 14:30:27 2022] DELETE /identity/v3/users/8fda79162e8e418698513f16d25884eb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:30:27.695238 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a56b2-650f-4bb2-99bb-73973ad6d178 admin admin] Authenticating user token {{(pid=80158) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:30:27.696684 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.fernet_utils [None req-2b0a56b2-650f-4bb2-99bb-73973ad6d178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80158) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:30:27.709717 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a56b2-650f-4bb2-99bb-73973ad6d178 admin admin] Validating token access rules against request {{(pid=80158) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:30:27.760822 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:30:27.768666 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0a5f5-a484-4ce8-bb52-7058bb991c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a10bd46534e415080cc3fb7cb29bb4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'defeadff0f9c417facade99f9905fca6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80158) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:30:27.770016 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0a5f5-a484-4ce8-bb52-7058bb991c80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:30:27.770328 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0a5f5-a484-4ce8-bb52-7058bb991c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:30:27.770870 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0a5f5-a484-4ce8-bb52-7058bb991c80 admin admin] PATH_INFO: `/v3/projects/347f076ba63f4ed69aecd3764c302f5b` {{(pid=80158) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:30:27.772662 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0a5f5-a484-4ce8-bb52-7058bb991c80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=347f076ba63f4ed69aecd3764c302f5b)` {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:30:27.774590 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0a5f5-a484-4ce8-bb52-7058bb991c80 admin admin] RBAC: Authorization granted {{(pid=80158) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:30:27.821556 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4dd0a5f5-a484-4ce8-bb52-7058bb991c80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '347f076ba63f4ed69aecd3764c302f5b', 'request_id': 'req-4dd0a5f5-a484-4ce8-bb52-7058bb991c80'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:30:27.826941 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4dd0a5f5-a484-4ce8-bb52-7058bb991c80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '347f076ba63f4ed69aecd3764c302f5b', 'request_id': 'req-4dd0a5f5-a484-4ce8-bb52-7058bb991c80'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:30:27.834332 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4dd0a5f5-a484-4ce8-bb52-7058bb991c80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '347f076ba63f4ed69aecd3764c302f5b', 'request_id': 'req-4dd0a5f5-a484-4ce8-bb52-7058bb991c80'} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:30:27.845544 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4dd0a5f5-a484-4ce8-bb52-7058bb991c80 admin admin] The token cache is being invalidate because project 347f076ba63f4ed69aecd3764c302f5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80158) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:30:27.845836 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4dd0a5f5-a484-4ce8-bb52-7058bb991c80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:30:27.846500 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG keystone.notifications [None req-4dd0a5f5-a484-4ce8-bb52-7058bb991c80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80158) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:30:27.848166 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: [pid: 80158|app: 0|req: 1461/2924] 10.222.0.94 () {58 vars in 1232 bytes} [Mon Jul 18 14:30:27 2022] DELETE /identity/v3/projects/347f076ba63f4ed69aecd3764c302f5b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:30:37.518477 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=45,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:30:37.842604 dv-cinder-849846-1-iscsi devstack@keystone.service[80158]: DEBUG dbcounter [-] [80158] Writing DB stats keystone:SELECT=26,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80158) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:32:44.805902 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bc013-0d13-4566-96c7-10c69af21469 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:32:44.807296 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-129bc013-0d13-4566-96c7-10c69af21469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:32:44.876251 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129bc013-0d13-4566-96c7-10c69af21469 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:32:44.933397 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7994d41-4585-4c82-b42b-7f22d1a038ea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd5b2ba9e37041c78a99f1700a4970c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec94946acf91486ba3f5b4fe834b8076', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:32:44.934044 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7994d41-4585-4c82-b42b-7f22d1a038ea service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:32:44.934328 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7994d41-4585-4c82-b42b-7f22d1a038ea service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:32:44.934606 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7994d41-4585-4c82-b42b-7f22d1a038ea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:32:44.936460 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b7994d41-4585-4c82-b42b-7f22d1a038ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:32:44.950533 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7994d41-4585-4c82-b42b-7f22d1a038ea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:32:44.952144 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7994d41-4585-4c82-b42b-7f22d1a038ea service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:32:44.961914 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1464/2925] 10.222.0.94 () {60 vars in 1365 bytes} [Mon Jul 18 14:32:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:32:54.957717 dv-cinder-849846-1-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=58 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}